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PREFACE 



The Rand Corp orat ion- -ts conducting , under the sponsorship of the 
U.S. Office of Education, a several-year, two-pha^e study of federally 
fuTided programs designed to introduce and spread innovative practices 
in public schools. These change agent programs normally offer temporary 
federal funding to school districts as **seed money.** If an innovation 
is 'successful, it is assumed that^the district will incorporate and 
spread part or all of the project using other sources of funds. The 
Rand study analyzes the effects theae federal policies have had on 
local change processes . 

The first phase of the research (July 1973 to April 1975) examines 
four federal change agent programs (E,lementary and Secondary Education 
Act Title III, Innovative Projects; Elementary and Secondary ■ Education 
Act Title VII, Bilingual Projects; Vocational Education Act , 1968 Amend- 
ments, Part D, Exemplary Programs; and 'the Right-to-Read Program) and 
addresses issues related to the initiation and implementation of these 
change agent projects. Specifically, this aspect of the study identi- 
fies what kinds of strategies and conditions tend to promote change in 
the school and which do not. 

The final phase of the research (May 1975 to April ^97 7) examines 
what: happens -to local projects in the two largest change agent programs — 
ESEA Title III and ESEA Title VII~when federal funding stops. This 
phase focuses on the different |ginns that local incorporation or con- 
tinuation, may take and. analyzes the institutional and project factors 
that promote. or deter the sustaining and spreading of Title III and 
Title VII projects. - 

The study's findings are reported in eight volumes under the general 
title Federal Programs Supporting Eduoational Change (R-1589-HEW) . A 
series of five reports* describes the results of the first phase of the 
research: 

Volume I (R-1589/1-HEW, A Model of Educational Change) provides a • 
theoretical perspective for the Rand study by analyzing the current 
state of knowledge of planned change in education and by proposing a 
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conceptual model of factors affecting change processes within school 
districts.*^ 

Volume II (^"-1589/2-HEU, Fao^tors Affecting Channe Agent Pr*oJeots) . 
contains the analysis of svyrvey data collected from a national sample 
of 293 projects 18 states; during November and December 1973. 

Volume? '^^-^ (H-1589/!^-HEW, The Py^oaess of Change) summarizes the 
findings and pd^^^y implications resultflng from 29 case studies of 
change agent pr^^J^cts conducted by Rand staff members and consultants 
in 25 school di^^^icts during April and May 1974. These case studies 
were chosen froni the original sample- of 293 projects initially surveyed . . 
Volume III also ^^scribes the role of state education agencies in 
selecting, manag^^^g, and disseminating the change agent projects. 

Four techni^^l /appendixes to Vol. Ill describe in detail, the 
federal program ^^nagement approach, . state edu^cation agency participa- 
tion, and case studies for i^ach of the programs in the study: Title III, 
App.A: Reading? ^po. B; Bilingual Educationj App . C; and Career Educa- 
tion, App. D. Appendix A should be' of particular interest to researchers- 
or practitioner^ Concerned with the introduction of new approaches to 
classroom inStrU^t'ion . 

Volume IV (^^-^ 1589/4-HEW, The Findings in Review) summarizes the 
findings of Vols* I, and III, and., also synthesizes extensive data 

collected by Ran^ on federal-level program strategy and management for 
each of the change agent programs. Volume IV also includes a discussion 
of alternativ/e f^^^ral strategies for promoting innovation. ' 

Volume V (I^"l589/5-HEW, Executive Svatrmary) summarises the first 
phase of the re^^^irch .for a general audience. 

The results the final phase are reported in three volumes: 

Volume VI (J^^l589/6-HEW, Implementing and Sustaining Title VII 
Bilingual Pr^oj^^^^^ » discusses the complex process of establishing 
bilingual progrB^^ in local school districts, with particular attention 
given to those aSPects of the Title VII program and to those political 
influences that ^^^ect local -implementation. The fieldwork, viewpoint, 

■^Because of ^^nd ' s interest in advancing knowledge of organiza- 
tional behavior educational institutions, the research underlying 
t his r eport was supported in part by an allocation of Rand corporate 
research funds. 0 
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and data interpretation build on the extensive empirical work done in 
the first phase of the study and reported in Vol. IIX, App. C (Innova- 
tions in Biliyinual Ecluration, R-1589/3-HEW) . 

The present report, Vol. VII, presents an analysis of t:he survey 
data collected in 100 Title III projects in 20 states. This volume 
deals specifically with the,, questions of implementing, sustaining, and 
spreading , part or all of special project strategies after federal 
support ends. 

^ Volume VITI (R-158978-HEW, Implementing and Sustaining Inn'02)ations) 
summarizes the findings from both phases of .the study and, drawing on 
these results , describes the process of change at the local level — 
initiating, implementing, sustaining, and spreading innovative projects. 
Volume Vlir.also includes a discussion of policy implications that 
derive from this study. 
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SUMMARY 



This sevcnt fj vohimo in tlu' change agent series prosonts findings 
,()(* tho st'cond phnst' of Kand's study of local innovation projects fund(?d 
hy federal change agetit prnf;rains. The study's first phase dealt with 
tlio Initiation and implementation of eliange projects.. The second phase 
focuses on the continuation of .innovations after ttie end of their 
federal grants and seeks to understcmd the long-run effects of the 
federal policy of stimulating local education refbrm by providing "seed 
money" to sc.hool districts. This study addresses thre^ " research 
qu es t ions : 

o Wliat does the continuation of spec ially funded projects mean 
and how s]iou*ld this process be assessed? 

o What influences the nature and extent of continuation at the 
classroom level? 

o' How do districts deal with change agent projects at the end 
of federal funding, and how do their actions affect the long- 
term fate of the projects? 

RESEARCH DESIGN . • 

The research consisted of 'a nationwide survey of 100 Title III 
projects one to two years after the end of their federal funding, 
fieldwork in 18 selected school d ist ric ts , and statistical analyses. 
The sample of projects, most, of which ^ere surveyed during Phase 1, is 
not representative sanple of all former Title 111 projects; it con- 
tains primarily projects that the districts have continued, at lea^igf 
to some extent. Because of the extensive data base collected during 
Phase 1, however, it is possible to compare the characteristics of dis-^ 
continued and continued projects.. Those in the sample cover -a wide 
variety of innovations adopted in a wide range of local settings. 

The survey queried 100 superintendents or key LEA officials, 
171 principals, and 1072 teachers. The superintendent's questionnaire, 
administered as a personal interview, focused on district decisions . 
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concerning the project's adoption and continuation. The principal's 
questionnaire, also a personal interview, elicited information on 
project design, the school's earl^'^ivolvemen t in the innovation, staff 
attitudes toward the project, and the project's effect on the school. 
Teachers filled out self-administered questionnaires that covered many 
topics, including the nature of the project — its implementation, its 
effect on teachers and. students, and its continuation — and the school's 
characteristics and organizational climate. 'The teachers* were also 
asked to take a short verbal ability test. 

The data analysis used statistical techniques to explore^the 
effects of,., the characteristics of projects and of the institutional 
setting 6n t*he continuation of innovations after federal funding * ended . 
To obtain comparable measurements of "project results across the array 
of innovations, we used teachers' assess;ments of project effectiveness 
(the percentage of. project goals achieved), the change in teaching 
style or behavior, and improvement in student performance in both cog- 
hitive and ^fective aspects. The measures of continuation at the 
classroom level were • teachers ' assessments of the extent of their use 
of projei!t methods and materials ,ap proximately two years. after the 
end of federal funding. At the district level, the primary "dependent 
variables*^ were the continuation status of change agent projects and 
the likelihood that the district would maintain the project. 

^ Because of the lack of theory about school district behavior and \ 
abou\ the local process of change, the researcVr aim was not to test 
hypotheses but to enable the formulation of hypotheses by developing 
a systematic understanding of continuation and the factors affecting 
it. Our operational measurements of both independent and dependent . 
variables can be challenged, as can-virtually all measurements of 
educational "inpyt** and outcome; the selectivity of our sample raises 
questions about the generalizability of the findings; and the statisti- 
cal procedures and the interpretation of the results are open to valid 
criticism and alternative interpretations. These caveats notwith- ^ 
standing, the findings provide working hypotheses for federal, state, 
and local policy. ' 



FrNDJNGS AT THE CUSS ROOM LEVEL 

. , _ — . ^ 

Teacliers' attitudes toward continuation depended on Jiow effectively 
the project was implemented and on how much they changed because of it:.. 
"Among the factors affecting continuation, it was therefore important, to 
examine those related to implementation and teacher change. We identified 
three categories ,, of such factors; federal input, project: characteristics, 
and institutional setting. The following are our major findings for each 
ciitegory: 

** ' I. 

Federal Input 

Differences in the amount of Title III funding had little consistient 
or significant effect on project outcomes or on continuation. Although 
federal money clearly mattered to local school districts in a number of 
ways — particularly, their ability to undertake a project at all-- any 
change wrought in district practices depended on what the district did 
with the funds, not on dollar amounts. 

Project Characteristics 

Educational Method. The educational method or technique of an 
innovation had little effect on project implementation, outcome, and 
continuation. Because of differences in project implementation strategies 
and in local institutional factors, sirilar methods produced dissimilar 
results across sites. Generally speaking, therefore, no one method was 

^h^" another to fulfill its promise, particularly in the 
long term. In terms of the ''success" of a federal investment, then, it 
mattered less what the project was than how it was carried out. 

Scope of Proposed Change. Projects attempting a broad scope of 
change were not more or less likely to break down, and were somewhat 
more likely to be continued, than were less ambitious or more narrowly 
defined projects. Indeed, projects that required Significant Change in 
overall teaching style and that called for extra effort on the part of 
staff induced significantly more teacher change. Ambitious projects 
therefore were not a "liability" from a funding point of view. Clarity 
of project, goals was important in the imp^ementat io'ti of all projects, 
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irowcvev, particularly lluiso ;U tiMnpL. i ng a broad scdpi^ of fliniij',o. h'or 
such pro)(»ct:s, staff unci» r fa In ty about: what: t liey were cxpcM-tctl to do 
generated severe implementation prol)loms and ,eont r Ibuted to project 
demi.se once federal fundinp, ended. Clarity of goals could not be 
"given" to a project staff at the outset, [lowever; it had to be achieved 
in the course of imi)lemen tat ion . 

l!in>Lrnu!>itiiLu.in iU.Vd f.r^jicr. . The implementation strategies chosen 
for a project strongly affected it.? outcome and Its continuation. In 
particular, well-conducted training in liow to use project metliods and 
materials enhanced implementation and improved student performance. 
(The quality of training, not the quantity, made a major difference.) 
Even good training appeared to have only short-term effects, liowever, 
unless it was given in combination with staff support activities. 
Specifically, when activities such as assistance in the classroom, the 
use of outside consultants, classroom obse rvat ion , and staff meetings 
were done effectively (especially by giving practical support), they 
provided the feedback and timely assistance the staff needed in adapting 
project strategies to their own classrooms. Such assistance also helped 
create tH'e climate and moral support necessary to motivate teachers to 
expend the effort that made projects work. The cruciality of well- 
executed staff support activities was evident in the strong positive 
effeccs they had, as a group, on all the dependent variables: percentage 
of project goals achieved, teacher change, improved student performance, 
and continuation of project methods and materials. 

Teacher participation in proj ec t decisior)S enlianced implementation 
and heightened the chances for continuation. It did so because^ It pro- 
moted a "sense of ownership," especially when the staff developed part 
or%il of their own materials. 

Pay for training did not contribute to continuation and, in fact, 
was negatively related to the percentage of project goals achieved and 
total student change. 

The local allocation decisions tliat are part of project implementa- 
tion — that is, the number of project schools and the funding level per 
student — generally had no significant relationship to project outcome 
or cont innat ion. (There were two exceptions: more concentrated funding 
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\)vv sLiultMit was posiiiviMy a.s.^oc'i aUuI with Improvinl sL'iuloiU porf orinnnce; 
aiul I lu» gifaliM* t lu' miml)t»r (if projei-l .scliools, llic lUjOicr was I lu» pro- 
pori ii>n i>f |)r()j(.H-t nwrOtM* i al . coiil imiud . ) 

Institutional Sultiii^ ^ 

i.^>\}.i>! I I 'i 1/ i'!>ntiitr ,:h<! //('(/(/t '/'.'r////'. Leadersliip was a villi 
I'actor a I ln>Lh tlie scliool and I lit* project ievei. KfTet^tive impiemen ta- 
t ion recjuired a good project director and a supportive school principal. 
But continuation depended less on having luid an effective project direc^tor 
than on t hi» e^ir ly and las t ing suppor t of principals. Tlie organiza t i ona 1 
climate of tlie project — the quality of working re la t ionsli ips — strongly 
affected the percentage o'' goals acliieved and project continuation. 

:-'(4uu'l i'h(i)'fi--f(n'tr,f.i<*i^. The f-ljpc of school liad little or no 
relationship to project outcome or continuation. The exception tq this 
general finding was the difficulty encountered by secondary school 
projects both in achi eving ,ef f ec t ive implementation and in promoting 
teacher change. 

T(\:i*k('i[' i''hava<* f.ppistitii\. Teacher characteristics had major effects 
on project outcomes. Above all, teachers* sense of efficacy emerged as 
a powerful explanatory variable; it had major positive effects on the 
percentage of project goals achieved, improved student performance, 
teacher change, and continuation of project methods and materials. 
TL'achurs* yoars of experience, in contrast, had a. consistent negative- 
relationship to project outcomes; experience was negatively related- to 
tlie percentage of project goals achieved, teacher change, and student 
improvement. Teachers' verbal ability was positively associated with 
improved student performance, but otherwise did not affect implementa- 
tion, teacher change, or continuation. 

FTMDINGS AT THE DISTRICT LEVEL 

The end of each federal Title III grant confronted school disfrict^ 
with a decision that many did not seriously plan for when they first 
adopted a project: They had to decide whether to continue a project in 



whole or in part, using district resources, or to drop it. The latter 
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clioLco w;is easy wlien tho project lintl not bt»L»n effort ivc*ly imp lemon ti'd . 
Typically, however, tlie docisLon to discontinue meant tlmt tlio dlstri'/t 
had originally taken on tlie project out of opportunism--more to take 
advantage of available federal funds tlian to deal with pressing eduea- 
tipnal needs — did not consider them central to tlie district's educa- 
tional objectives, and gave them little institutional sujjport during 
Implementation. 

In contrast, the decision to continue was more difficult and com- 
plex, because it called for a district commitment in financial, organiza- .. 
tional, and political terms. The end of federal funding usually brouglU 
sharply reduced project budget^, particularly for expensive projects. 
Innovations that had used soft money to lower the student- to-adult 
ratio (e.g., by hiring aides) were cut back to live within tlie district 
budget; projects that had spread theiV resources thinly across botli • 
elementary and secondary grades were severely reduced in scope. In most 
'cases, districts did not provide* supplementary funding for '^continued** 
projects, which stayed in a ^'special" status rather than being incor- 
porated into the district's regular educational practices. Two years 
after the end of the federal grant, "they were still vulnerable to 
financial, organizational, and political • instabilities . 

Most continued projects followed either one of^ two patterns. First, . 
some projects were isolated^ in that district administrators simply left 
the projects up to school-level staf f . .The coatinuation of project- 
related changes in this case depended on the extent to which the -project 
staff had assimilated project precepts and integrated them into their 
classroom practices. Such isolated changes face a precarious future, 
threatened by staf f • turnover , principal transfers, and financial pressures. 

A second pattern of continuation was pro forma^ in that district 
officials formally decided to continue the project but teachers did not 
use project-related activities extensively in their classrooms. This 
pattern came about in several ways, the most frequent cause being in- 
appropriate or inef f ective '^district-level continuation strategit :. For 
example, when ^district officials simply "mandated" the installation of 
project activities in new sites, without first eliciting the support of 
staff new to the project, both teacher resistance and inadequate teacher- 
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■ preparation often made project operations only superficial. Or, when 
district administrators failed to furnish the support necessary to 
maintain project-related change — securing the commitment of principals 
or supplying materials, for example — teachers were not able to continue 
the project fully. 

In contrast to these unstable patterns, a minority of projects, in 
our sample became institutionalized — that is, project-related change 
became integrated into regular operations at both the district and 
classroom levels. These projects shed their "special" status and 
r*eplaapA practices that existed before the projects began. The key to 
effective continuation is for district officials- to see institutionaliza 
tion as a process of "remobilization" and "reimplementation- " They must 
realize that the perpetuation of a change agent project requires the 
early, active, and continued attention of school district managers. 
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A liirgc'-scalc resoarrli effdrt relies on the talents and inter- 
- (i ise ip I inary skills of many people. We were most fortunate in Iieing 
■ able to draw on the t-xpertise of an extraordinarily dedicated group 
of professional researchers. C;a i 1 ISass and CJail Zellman wc;rked tire- 
lessly to design, pretest, und revise the survey instruments; the 
critical co.mments of Mimi Baer and Rii:liard Murnane contributed greatly 
lo their efforts. Linda Prusoff and Joanne W<ichitech skillfully con- 
ducted a telephone survey of candidate projects. The Response Analysis 
Corporation efficiently administered the survey questionnaires thanks 
to the efforts of Me 1 ody Todd, and .James Ftiuss; Kdward Pauly. collabo- 
rated- iii all phases of the study, particularly in managing (with. Brent 
"A'lciwcf) our complex data set, and .conduc t i ng tlie stat ist ica 1 ana lysis 
for App. B; he avoided \^riting this report only by returning to aca- 
demic I i f e . 

"The analysis r^'ported' here has been improved hy the comments and 
substantive expertise of Rand co 1 leagues Steph(-n Carroll, .lohn Pincus, 
.lohr. Kol[)h, and Daniel We.iler. Tora Kay Bikson and James Hosek, who 
served as technical reviewers for this report, offered especially per- 
{•eptive and useful suggestions concerning the presentat ion and inter- 
pretation of this materia,!, Ann Bezdek, the U.S. Office of Eduiation 
Project Officer for this work, has been particularly supportive and ' 
patient during the course of this research; her comments have substan- 
tively improved thc^' study design and data analysis. 

Constance Smed ley and Rose-Mar i e V igi 1 handled the tedious job of 
manuscript preparation with notable skill and gdod humor.- The editor- 
ial talents of Wi I 1 Har riss , of the .Rand Pub 1 i4:ia t i on s Department ,. he 1 ped 
turn our. rough draft into- a finished product. 

Finally, we would like to acknowledge the contributions of the 
many superintendents, principals, and teachers who so graciously gave 
of their time and insight. In par t i cu 1 ar., we wo'uld like to thank 
Harry Handler (hos Angeles linified Scniool District), (Jerald Roseander . 
(Fresno Linified School District), and William Stegeman (San l)iegc\ 
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Unified School District), who helped us interpret the i n f o nriii 1 1 on 
lected from public school personnel ^3 round the country and to put 
into perspective. 

All of these contributors have facilitated and improved this 
port without being in any way responsible for its shortcomings. 
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Chapter 1 
INTRODUCTION 



This report presents findings of the second phase of Rand's study 
of local innovative projects funded by federal, change agent programs. 
Jts. purpose is to explore the long-run effects of the federal policy of 
providing "seed money" to school districts as a stimulus to educational 
reform'T^ To thi^ end, we examined change agent projects one to two years 
after the end of federal funding. 

BACKGROUND OF THE STUDY ^ • 

During the 1950 ' s • and ■ 1960 ' s two important initiatives — the National 
Defense Education Act (NDEA) of. 1958, and the Elementary and Secondary 
Education Act (ESEA) of 1965 — defined a new federal role in local educa- 
tion. NDEA sought to s t imulate curricu lum development in subjects such 
as science that are directly related to the national interest. These 
efforts were pursued m^inl^^ through colleges and universities, not local 
school- districts. ESEA initiated a much broadet federal role, including 
large-scale federal support for special education for the disadvantaged 
and for bilingual education, encourageirient of innovations in . the public 
schools, and grants to strengthen state department's, of education. ESEA 
also was intended. to serve social and political goals, such as redis- 
tributing educational resources in the public school system and legiti- 
mizing the voice of parents and communitx^-'grcjups in. the governance of 
the public schools. ESEA , -in shor t , signified an explicit federal 
interest in the r^eform of the ..nation ' s publ ic schools. 

The f ederaTgovernment , particularly dhe United States Office of 
Education (USOE) , has pursued a number of pol icy. strategies to promote 
educational feform. Cases in point a^re such ' programs as Title III of 
ESEA (innovative pVbgrams) , the experimental schools program. Right to 
Read, FollQw Through, and Title VIT of ESEA (bilingu-al education). 

One. -consequence is that federal funds now make up an important 
fraction of many local school: district budgets, but their effectiveness 
in improving ^local educational practices is uncertain. Federally 
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sponsor,e<^ evaluations reveal inconsistent and generally disappointing 
results."^ , 

Despite considerable innovative activity on the part of local 
school districts and despite some successful, indeed nationally vali- 
dated, projects, the evidence suggests that; 

o No class of treatments has been found that consistently leads 

i 

to improved student outcomes (when variations in the insti- 
tutional setting and nbnschool factors are taken into account) • 

o "Successful" projects have difficulty sustaining their success 
over a number of years, 

o "Successful" projects are not disseminated automatically or 
easily, and their ''replication" in new sites usually falls 
short of .their performance in the original sites. 

Consequently, 'although f ^deral support for local school services has 
become well established, the "decade of reform" that began with ESEA 
has provided few clear lessons about what went wrong (or right) and 
thus offers little guidance as to what an appropri.?te and effective 
federal role might be. 

To aid in reexamining and redirecting federal education politics, 
USOE awarded a contract to The Rand Corporation in 1973 to undertake ^ 
a four-year study of innovative projects funded by specified federal 
change agent pi^-ograms. This research was to assess the ef f iictiveness 
of these prografns as stimuli of change in local practices and to suggest 
how federal policies could be improved. The study was not to be an 
evaluation per se; it was to concentrate on the basic processes that 
attend the local initiation of federally sponsored projects and attempt 
to undetstand what factors systematically and signif icantly. affect this 
process . 

"^Volume I of this study reviews the evaluation and other, relevant 
literature up to 1974. Large-scale evaluations are a continuing activ- 
ity of U.S.. Office of Education (USOE) and National Institute of Educa- 
tion (NIE) . Recent evaluations seem as controversial and as mixed as 
their predecessors. On balance, however, they do not jseem to contradict 
the generalizations made above. . 
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In particular, USOE wanted Rand to examine the following questions: 

o How are federally supported innovations in local educational 
practices introduced and implemented? 

o How are these innovations sustained and spread after the end 
of federal funding? 
^ o What factors in federal policy, in the nature of the change 

, age^t projects, and in the local institutional setting deter- 
mine the outcome of innovations and their chances of being 
sustained and spread? 

FINDINGS OF PHASE T ; 

To address these basic questions. Rand conducted a two-phase study. 
Findings from the* first phase of the research are presented in the 
first five volumes of the report, yederat Programs Supporting Educa- 
tional, Change y R-1589-HEW.* The second phase is the subject of this 
volume.' 

Phase r (July 1973 to July 1974) studied local innovations during 

their last or next to last year of federal funding. It focused on 

project initiation and .implementation . USOE specified four programs 

for inclusion in Phase I: Elementary and Secondary Education Act. 

Title III, Innovative Programs; Elementary and Secondary Education Act 

Title VII, ^^ilingual Projects; Vocational Education Act, 1968 Amendments, 

2 

Part- D, Exemplary Programs; and,.Right to Read. These programs- differed 

,in focus and management strategy, but had a common purpose: , to stziniu- 

" c-'. 'V' ■ 3 

late and. disseminate educational innovations. They also had a coimrion 



2 • •" • 

In FY 1974, the year of the major- research activities of Phase I 
of this study, the approximate funding levels of the federal programs 
were: ESEA Title 111^ Innovative Projects, $150 million; ESEA Title VII, 
Bilingual Projects, $45 million; Vocational Education Act',' 1^8- Amend- 
ments,. Part D, Exemplary Programs, $16 million; and Right to Read, 
$12 million. ^ Although ' these programs have evolved further since 1974 
(e.g., Title III has t)ecome consolidated into Title IV under the 1974 
Amendments to ESEA), the discussion in this report refers to the pro- 
grams as they existed when the projects were studied. 

Each change agent program had a distinct focus and management 
strateigy.'' The largest,, Title III ,^ was' designed to improve the quality... 
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policy instrument: the provision of temporary funds (3 to 5 years) 
ranging from less than ten thousand dollars to several hundred .thousand 
dollars per year. Although these amounts are small relative to the 
usual school district budget, they were Intended to func}^ new educational 
services, not to support existing ones. Because we studied four programs 
instead of one , we were able to examine the local process of change in 
approximately 300 projects in a Variety of institutional settings, and 
identify principal factors affecting project initiation and implementation 
The major conclusion reached in Phase I ca^ be summarized simply : 

Federal change agent policies had their primary 
effect on the initiation of projects but . . 
neither those policies that were unique to each 
of the federal programs nor those policies that 
were common to them^had a strong influence on the 
implementation of local innovations". Federal 
■ change agent policies exercised limited leverage 
on the course of innovations because they did not - 
critically influence those factors most responsible 
for effective implementation- . . . ^ 



of public education both by introducing model practices that were new to 
American education and by spreading existing successful practices, to 

" schools that were not aware of them. The competition for Title migrants 
of three-year duration was open to almost^ any kind of project that local 
schools wished to propose. In 1973-197A, the first year of the Rand 
study, 15 percent of Title III money was granted directly to local educa- 
tion agencies (LEAs) by the O'ffice of Education, and the remainder " was al- 
located to state education agencies (SEAs), who in turn made grants to 
LEAs. Other federal programs were more narrowly targeted and had more 
specific f und ing" criteria . The USOE's Right-to-Read. program strove to 
create a national educajtional priority for readln'g, particularly for d'is- 
advantaged students. The Righ t- to-Read demonstration projects, the' 

J program component addressed in Phase I of. this study, included .a pre- 
scribed planning and management strategy in an' effort to facilitate -ef- 
fective implementation. Vocational Education, Part D, was designed to 
create exemplary programs to enhance career awareness and readiness. 
Congress, be.lieving that many SEAs were unable to promote significant 
innovations, authorized USOE to fund local projects directly through 
the Part D program. .Title VII (Bilingual Education) originally sought^ 

. -to provide model projects for the special needs of children whose 
English-speaking ability was limited. The program has subsequently 
acquired the further goal of maintaining and encouraging "cultural 
pluralism" in American public educa t ion , ^ wi th strong political suppbrt 
from many Spanish-speaking people.' , 

^R-1589/4-HEW, p. 2*4, hereinafter cited as "Vol\. IV," as other * 
volumes will be cited simply by number; "Vol. I, VolSs'II," etc. 
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Three main factors affected the local process of change and thus 
the outcomes of innovations: 

o Characteristics of the institutional setting, particularly the 

i>rj;an i L i ona 1 rlimato in schools'and the motivaLions of par- 

t icipanls at all l.evc»ls of the district; 
o The implementation strategy employed by local innovators to in- 

stall the p ro j ec t t re a tnien t ; and 
o The .scope of ch§nge planned or implied by the project relative 

to its setting. 

* , ** 

The mere adoption of "improved" educational methods., practices, or 
technologies did not automatically improve student outcomes. The reason 
was. that "project implementation j which is largely shaped by the insti- 
tutional setting, dominates the innovative process .... This means 
that initially similar, tiechnologies installed in different settings 
undergo unique alteration and thus t|ieir outcomes cannot be predicted 
on the basis of' treatment alone. 'in short, internal 'factors and local 
decisions are more influential than "inputs" from the outside. 

One implication, is that federal policy should pay more attention to 
the local implementation process. (.USOE ^and NIE have taken steps to do 
so.) Most projects that did not work failed either because school dis-,j 
tricts did not take them seriously — for example, when the major motiva- 
tion was simply to take advantage of federal funds instead of to solve 
a pressing educational, p^roblem — or because school districts intent on 
reform nonetheless lacked the institutional., capacity and skills to carry 
out an effective project. The priim^y feature of effective implementa- ' 
tion could be called "mutual adaptation," in which the project is adapted 
to its institutional context a^i^3? organizational patterns are adapted to 
meet the demands of the project. Phase I identified, in an exploratory 
way, local implementation strategies that promoted mutual adaptation.^ 
A. major objective of Phase H was to explore t^iose strategies mgre fully 



^Ibid. 

^Ibid., pp. 18-20. 
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and to consider how federal policy might assist school districts to 
develop mutual adaptation/ 



^ RESEARCH QUESTIONS OF PHASE II 

Phase II (May 1975-April 1977) treats questions that are in some 
ways more perplexing than those addressed by Phase I. USOE wanted to 
know what happens to innovative projects after the end. of their federal 
funding periods (normally three to five years). Accordingly^ two years 
after the initial field research. Rand revisited or resurveyed a sample 
of approximately 100 of the projects funded under Title III of ESEA,. to 
explore the longer-run effects of providing federal "seed money" to 
promote educational, reform. 

A major challenge in dealing with this issue was to pose the" 
"right" questions. The question usually asked is: Are projects 
"continued"' after. the end of federal funding? This question', though 
appropriate, is' far too simple. It assumes that when federal funding \- \ 
terminates , districts make a "go or ho"" go" decision about the^'fate of 
the project. Sometimes they do, as our inve^s.tigations at the end of 

the first phase, and the beginning of the second phase of the study 

' 8" ■ * ■ . •■• - ■ ^ ^ 

confirmed. Y.et »we found that such decisions were usually foregone 

* '• '■ ■ ^ ' 

conclusions, made because projects had not really been implemented, ^ 
often biroke ,down„ completely^ Qi^.^were at best symbolically implemente/I. 



7 V 

As will be discussed in a later' section, Rand gathered information— 
by a telepl|one survey, field visits, or a full-scale field survey--on 
over 200 projects, of which about 15 percent were not in the. original 
sample. According to an agreement with' USOE, the final selection for 
quantitative analysis amounted to ioO former Title III projects, and 
former Vocational 'Education and former Right- to-Read projects were not 
studied. Moreover, Title VII, Bilingua^'l projects were not included in' 
the quantitative analysis because a change in Title VII funding* policy 
meant that most (85 percent) of the projects in Rand's original sample 
received additional federal funds. Nonetheless, since the isSue of how 
biliiigual projects might sustain their activities is still important, 
these projects were examined using qualitative methods; the results are ■ 
reported in Vol. VI. 

8 ^ 

In Phase I we asked district officials and project participants 

whether they thought the project would be continued (see Vol. II, 
Sec. '5). Phase II did preliminary •fieldwork in^'a small number of sites 
followed by e telephone survey of Tittle III projects involved in Phase T 
research^ Chap. 2 below discusses this preliminary research. 
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Such failures both to implement and to continue tended to bie. predictable 
. from the outset, especially for districts whose apparent motivation was 
the opportunistic desire to obtain federal money. Not surprisingly, 

such projects tended- to disappear almost without a trace when federal 

9 ^ 
funds ended. . . ^ 

Districts interested in project continuation face a more complex 
problem than a simple yes-or-no decision. District staff have to figure 
out what components of a project are worth sustaining and spreading and 
on what scale., given the district's economic, .political, and 'organiza- 
tional realities. Questions abound. Has the project proved its worth 
and' therefore should become a regular part of the district's educational 
^>rograra?^ ShoulH project activities be cut back, remain the same, or be 
expanded? Should project methods be supported but project materials be 
dropped? Or vice versa? What should happen to project personnel — the 
project director, the staff, and the aide.^? Should new people and new. 
sites be brought in? A major objective of Phasgi' II was to examine what 
proje'cl:, economic ,. and institutional factors systematically affected the 
choices that school districts made. 

Phase I, however, had previously found. that school district deci- 
sions are seldom implemented automatically or exactly as planned, and 
that another level of decisions also influences" what -Ijagpens to a project 
The district's ability to sustain or spread innovatiQn|i^-ijdeRends h 
on people at the school and classroom level. 

The actions of teachers and principals may or may not reflect dis- 
trict level policy. This problem cuts two. ways. For example, individual 
teachers may assimilate all or part of the practices of an innovative 
project independently of the formal district view of the project or, 
indeed, of the awareness of district officials. ' Because such assimila- 
tion can be both enduring and significant, it needs to be weighed as 
part of a project^s lon3;-term effect. Conversely, formal district 
decisions to "continue*' a project may be meaningless if , either because 
of insufficient training or individual, preference, classroom, teachers 



The disappearance was 60 complete in a number of the sample proj- 
ects that our telephone survey conducted one year after the end of 
funding. had difficulty even locating project participants. 

. 27 ■ . • ' 



-8- 

t 

•a 

and principals do not follow district directives. It is crucial, then, 
to assess what really happens in classropms and schools after the end 
of federal funding. 

At this level, we are more concerned with a project's continuing 
effects on the everyday behav ior of the staff, than with the presence or 
absence of a formal project structure. To what extent do teachers con- 
tinue to use the project's methods and materials? Have teachers or . 
schools changed the way they operate as a direct or indirect consequence 
of the project? How do these changes affect student outcomes? Phase II 
addresses, tliese questions gnd seeks to identify characteristics of the 
j5*roject, the staff, the schools, and the school district that influence 
« thn longer-term effects of an innovation. ''^ 

... The dual level^of effective- decisionmaking in school districts^ 
presents yet anotlier implication for analysis. District level decisions 
to continue a project are not self-exelruting; they depend on decisions 
and preferences at the school and classroom level. Consequently, the 
analysis must be concerned with both the district level decision and 
with thei' strengths and weaknesses of the istrategieij selected by local 
officials to carry out their decision. For example, how can enthusiasm 
and commitment for a project best be generated at a new* site? What 
support is necessary to sustain project activities at the original sites? 
How can the. district best deal with the inevitable turnover or reassign- 
ment of the original project teachers and administrators? PHase II ex- 
plored these questions not ^ . because district officials need to fjnd 
ways to sustain and spread innovations, but also because a key to improving 
ciiange agent policies might lie in understanding ways in which districts 
can help themselves to continue changing after federal money goes away. 

In summary, the findings of this repor.t apply to the following 
questions: 

o What does the ccitinuation of specially funded projects mean 
and how should this process be assessed? 
I o l/bat influences the nature and extent of continuation at the 
. classroom level? 
o How do distri-cts deal with change agent projects at the end 
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of federal funding, and how do their actions affect the long- 
term fate of the projects? 



Chapters 2, 3, and A discuss the research design of this study . V .. 
Chapters 5 and 6, respectl veily , present findings of the classroom level 
c'.nalysis and of the district level analysis. 
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Chapter 2 

RESEARCH DESIGN AND CONCEPTUAL APPROACH 



To examine the fundamental questions posed by USOE, Rand designed 
a research plan for Phase II that comprised several methodologies: a 
nationwide survey (using telephone interviews, personal interviews, and 
self-administered questionnaires) of former Title III projects; field- . 
work in selected school districts; and statistical analyses. Using 
Phase I findings and the relevant literature, we developed a conceptual 
framework for analyzing the continuation ^ of federally supported change 
agsnt projects after the end of their funding. < This framework guided 
our data collection and ^analysis and enabled us to translate broad con- 
cerns into operational measurements. This chapter discusses the Phase II 
research design and presents our conceptual framework. Subsequent chap- 
ters will describe data collection and analysis and the selection and 
measurement of the variables used in the analyses. 

THE RESEARCH PESIGN 

When we began this study, information about how school districts 
implement and sustain change consisted .primarily of anecdotal evaluations 
or highly aggregated input-output analyses. Policymaker^ could not use 
such information because it was either too particularistic or too ab- 
stract. Consequently, our initial research had to be exploratory. Hy- 
potheses could not be tested; they first had' to be generated,. Moreover, 
the research design faced a "chicken and egg*' predicament; typical of 
exploratory research: On the one hand, to gather information we needed 
a systematic understanding (i.e., a crude theory) of the organizational 
realities of school districts; on the other hand, information about school 
district innovative behavior was a prerequisite to developing that system- 
atic understanding. 

To deal with these problems. Vol. I. formulated a crude framework or 
simple model of the innovative process based largely oh the social science 
literature about how organizations work. ■ This literature also allowed up 
to identify a large number of variables that might affect innovation. 



The research of Phase I examined many of these variables and inten- 
sively analyzed the early stages of the local process of change. We 
were thus able to narrow the. original mass of variables down to a 
^mailer number of crucial factors. Phase II therefore started out 
with an empirically grounded conception of how local innovations be- 
come implemented , hypotheses about a series of factors that might" 
significantly affect the fate of federally funded projects, and an 
extensive data base, 

Howe^^er, "testing hypotheses" was neither our goal nor our 
methodology. The research of Phase I could not deal with continuation 
in more than a preliminary way because the projects we examined had 
not yet completed their period of federal funding. Therefore, we 
still needed to develop theory about how innovations are sustained, and 
we needed to generate hy.potheses abouc what happens after the end of 
fed(?tal funding. Moreover, though the measurements of independent and 
dependent variables used in Phase I fulfilled their initial purpose of 
locating factors that matter to the outcome of change agent projects, in 
the second phase we had to formulate' more refined and focuse'd instruments 

Accordingly, in the overall plan of Phase II, we dreW:6n the ex- 
perience of the first phase to: \ . . ' 

• 1. Formulate an empirically based theoretical conception of con- 
tinuation and the local process of change; 

2. Identify factors that might affect this process and thus the 
outcomes and continuation of innovative projects; 

3. Collect data using the theoretical conception of the process 
of change. as a framework and guide; 

4. Specify a data analysis m-'del that would posit relationships 
among measures of a project's impact and the various factors; 

5. Analyze and interpret the data to provide guidance for 
policymakers and formulate a better theoretical understanding 
of innovations. 

The remainder of this chapter deals with, the first two 'tasks: our 
theoretical framework and the identification of significant factors 
in the change process. Chapter 3 describes the data collection activ- 
ities and the sample of innovative proj ecfs ; "Chapter 4 presents the 

3i ■ ^ 



■classroom-level data analysis model and discusses how we measured the 

effect and cont inuat ion.of the project. The findings of this phase' 

of the research are presented in Chaps. 5 and 6 and summarized in' 

Chap. 7. ! 

THE ISSUE OF CONTINUATION 

Our earlier research formulated an empir ically ^grounded conception 
of the process of change' and challenged several common approaches or ^ , 
assumptions abou.t how federally supported local change agent projects 
worked or could .be' j^iade to work. Two paints of view seemed particu- 
larly unrealistic. ^vF'irst, much of the social science literature fo- 
cuses on the adoption of innovations and virtually ignores implementa- 
tion; the implicit assumption seems to be that innovations are self- 
executing — that once adopted, better technologies . invariably produce « 
better outcomes. Corroborating the work of an increasing number of other 
researchers, we have found that implementation,' not adoption, dominates 
the outcome of innovative projects and that no educational technology 
automatically fulfills its promise. Accordingly, our Phase I research 
described the process of implementation and examined the factors that 
affected it. . ,v 

Our Pbase I research also challenged a second common assumption 
about innovations: that local schooV districts either did or, should use 
a cost-benefit calculus in deciding s.about adoption or implementation, 
in thjB manner described by students of industrial research and develop- 
ment. We found that districts rarely used such an approach. Moreover, 
our description of implementation as essentially a complex, multilevel 
organizational process led us to believe that an R&D approach would not 
work. We hypothesized that the key to effective implementation is the 
adaptation of the project to 'its institutional setting and vice versa; 
the R&D approach seems ill suited either to describe such mutual adapta- 
tion tor promote it. 

The issue of continuat ion has . received perhaps even less attention 
^ ■ ■ . '1 • 

from theorists and analysts than the implementation of innovations. 

Aside from the important concept of routinization, the social science 

literature has little to say about what happens to innovations when 



they are no longer new or *\special'* or experimental. Similarly, the 
"seed money" approach in federal policy either simply ignores the issue 
or. assumes a pseudora^t ionalis t ic model of school district behavior — 
namely^ that once an innovation has gone through its federally supported 
trial stage» the district wil] discard ' those innovations that failed and 
adopt those that worked. " 

A cursory examination of continuation reveals that model to be both • 
simplistic and inaccurate. For example, a study by the Ford Foundation- 
suggests that even successful innovations have a way of disappearing 
after several years. ^ Many anecdotes in the field also tell of impres- 
sive, innovations "burning themselves out." We need not recount these 
tales to make the point that innovations are not "self-sus taining. " 
It cannot be' assumed, then, that even when federal "seed money" leads 
to successful projects , thiey automatically "take root." 

Similarly, one cannot assume that school districts use a cost- 
benefit calculus in deciding whether to continue or drop a project. 
The termination of federal funding does indeed represent a decision- 
point for school d^istricts. When the soft money is gone, district 
managers must decide whether to turn the project off or support it- with 
local f.ynds. To some extent they base their decisioti*'on the educational . 
value of the innovation, but, that is not necessarily the most important 
factor* For example, we observed bilingual projects that . would have to 
be considered ineffective under most definitions of educational effec- 
tiveness, but districts con t iniiied them anyway . . Clearly, political and 
bureaucratic considerations outweighed short-run cost-benefit measures 
of "success." Conversely, we have seen ef fee t ive ^Droj ec ts abandoned by 
districts, even when' money or questions, "of other educational priorities 
did not seem to be a real issvie. In short, just as it is misleading to 
think of adoption of innovations in R&D- terms, it would be -similarly 
specious to approach, the dis trict ' s decision about continuation in 
pseudorationalist ic terms. 



The Ford Foundation, A Foundation Gdps to Snhool: The Ford 
Foundation Cornprehensive Schpp L JmproveiJieyit Pvorjrtvn 2 DGO'-l 9?.d'.:,x.0lVicv 
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Extra soft money was usually made available by federal or stat^ 

funds for bilingual projects. iSee Vol. VI for a discussion of bilingual 

proj ects . " . . 



^Moreover, the district's decision about continuing or dropping a 
project is just- that — a decision. Decisions are seldom self-executing 
in any organizational context, let alone a school district. We have 

e 

seen cases in which teachers "continued" a project in -their classrooms 
despite a formal district , decision to drop it. Conversely, we have 
seen teachers ignore or pay mere lip service to innovations mandated 
by official policy to be u3ed in every classroom. In short, researchers 
need to look at what actually happens at the classroom level. 

DEFINING CONTINUATION 

' 7. T 

The value of an innovation ultimately' hinges on one question: Are 

the school district and its students better off for having adopted an 

innovation? Unfortunately, this question is too basic to answer, at 

least by Systematic analysis. It has to be (divided into manageable 

pieces that necessarily fall short of supplying a single ainSwer. Two 

more specific but narrower questions help to define a research agenda: 

What wete the effects of a special project? Will those effects continue 

i'' ... 

over time? 

A central consideration*, of course, is change in student perfor- 
mance; but for th^t change to be attributable to a project,, the project 
had to involve some change in what was delivered to students or the way 
it was delivered. In other words, tciachers had to have changed their 
regular teaching style or behavior. Measuring "change" is. a difficult 
operational problem. Conceptually, howeveir, we simply mean that a 
teacher's regular pattern of behavior (including the use of technology 
and curriculum) at time t^ is significantly different from the teacher's 
regular pattern of behavior at time t^. Defining the key concepts of 
"regular pattern of behavior" and "significant difference" in operational 
terms is a challenge no researcher has fully solved. Our concept of 
change subsumes three types of alterations in the teacher's regular 
pattern: adding a new behavior, modifying existing behavior, or 
replacing old behavior, ■ ^ ^ 

A' change in a staff's classroom practices thus constitutes an ef- 
fect of the project; but unless it persists, the project will of course 
have no effect on future students. We ^ th fire fore dfi fine the continuation 
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of a ppojeat in temiG of the pevQistcnae of its effeats-'-that is^ the 
pevsistenac of the change in teaching pat]terma or in the delivery df 
service caused^ by the project. (\ 

Persistence of ett^6%^^^J^ not the same as pa±sistence of the proj- 
ect itself. Innovations, particularly . effective ones, mutate during 
implementation; the implemented projiBCt can differ sharply from its 
original form. Even the term "implemented project'*' is a misnomer that 
should read "the implemented project as of time t^." We would expect 
the project' to evolve continuously as both a reflection of and a means 
for its persistence. Therefore, by persistence,- we do not mean mainte- 
nance of the status quo of the project after the end of federal funding. 
At the end of federal funding, the district and the staff may further 
alter the project — e.g., drop some components, add new schools, reallo- 
cate personniel, streamline procedures — so that the project bears even 
less resemblance to its original^ design . Thus the concern of federal 

and state of ficials • about the continuation of the adopter/ project seems 

3 

to misspecify the problem..' The real concern should be to figure out 
how to sustain, spread, and improve the desirable effects* of the project. 

Continuation of a project's impact is not the only concern of fed- 
eral or state officials, however. Specially funded projects can produce 
worthwhile results even if they are discontinued. We have seen several 
projects abandoned because school officials and staff felt the innova- . 
tions did not "pan out," but a healthy consequence has been that the 
staff and middle management learned what they can and cannot do. ' ' 

For example, one project we observed consisted essentially of the 
application of a packaged reading program, this project "failed" partly 
because it did not allow for mutual adaptation during implementation and 
because teachers did not develop a sense of ownership and commitment. 

~ 3 • 

Se^e Jerome B. Brightman-, "The Continuation Rate of Three-Year 
ESEA Title III Projects," A Report to the President's National Advisory 
Council^ December 15, 1971; Nopnan E. Hearn, "innovative Educational 
Programs: A Study of the Influence of Selected Variables upon Their 
Continuation Following the Termination of Three-Year ESEA Title III 
Grants," Ph.D. diss., George Washington University, Washington, D.C., 
1969; and Anthony N. Polemeni, "A Study of Title III Projects after - 
the Approved Funding Periods," Ph.D. diss.. School of Education, St. - 
John's University , Jamaica, New York, 1969. 
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The projectTwas terminated, but school officials and teachers learned 
some hard lessons about implementation. Two years later the district 
adopted another reading program, and found that the early failure had 

paved the way to a more effective project. 

\^ 

How does one predict whetl^er project-related change will persist? 
One way is to examine the status of the change two years after the end 
of federal funding — the empirical point of reference for this study. 
This approach makes sense because many projects are quickly dropped, 
either at the close of federal funding or perhaps a year later. How-, 
ever, though the rate of decay slows down after a year or two, the 
remaining innovations-— those not weeded out — can still" fade away. This 
calls for a conception of persistence that, acknowledges the continuing 
vulnerability of even successful projects. A one-year survival is hot 
enough. ^ 

Accordingly, we assume that proj ect-relate,d change will persist 
only if it is no longer vulnerable to being dropped, and that this stable 
state occurs when the change has become integrated into the school system 
as a standard or charaoteHstio feature of -.the district's operations^. 
For the sake of ^providing a single term to denote this cumbersome defi- 
nition, we use the word institutionalized. In short, we assume that 
project-related change will persist insofar as it becomels institu.tiona- . 
lized. (But our definition does not imply that an institutionalized • 
change is itself not subject to change.) Therefore, the Phase II re- 
search collected and analyzed data with the objective of understanding 
why and how educational innovations become institutionalized. 

CONCEPTUAL APPROACH TO STUDYING INSTITUTIONALIZATION ^ 

To gu^ide our data collection and analysis, we formulated a conceptual 
approach based --on two premises i. v 

1. Institutionalization is one phase of a complex organizational 
process of change . . 

2. Institutionalization occurs at two ' levels— -the individual 
teacher and the school system — and the process is^dif f erent 
for these levels. 

The process. of institutionalization can lead to a variety of 

'\ 
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consequences, including the continuation of the project at the same 
level, a reduced level, or an expanded level; organizational effects 
. (e.g'. , the reassignment of personnel or the reallocation of resources); 
and political effects (e.g., parent satisfaction or disaffection). From 
the outset of the research, we hypothesized that the extent and nature 
of continuation* depends on lu>h) institutionalization t^ikes place. Field 
observations made it apparent that institutionalization cannot be under- 
stood without taking into account the "history" of the project. We 
repeatedly saw that what happened early in a project profoundly affected 
what happened later. Institutionalization must be seen as but one phase 
in a complex local process. of change. 

Figure 2.1 is a schematic diagram of a model of the local process 
/of change that ensues ffom the adoption of an innovation. The model 
assumes that the course of a local innovation — including its continuation — 
results from, the interplay between a constantly evolving project and an 
inf^ti tutional setting itself subject to chaqjge prompted by the innovation 
or by a variety of internal and external forces. 

The process can be analytically separated into three phases — 
mobilization, ' implementation, and institutionalization — that are closely 
interrelated in practice. Thus,, some activities and decisions associated 
with each phase flow chronologically, as Fig. 2.1 illustrates. For ex- . 
ample, mobilization includes the inception of the project in the district; 
implementation includes the translation of the innovation into an opera- 
tional reality; and institutionalization includes the transformation of 
the innovation from its special project status. In addition to this 
linear sequence, the phases are interconnected by complex and not well 
understood feedback relationships, symbolized in Fig. 2 . 1^ by the double 
dashed arrows joining the three phases. 

4 

This model, d iscussed in more detail elsewhere, guided Rand's data 
collection and analysis. In particular, we used the model to identify 
and collect data on factors that potentially influence projects* effects 
and continuation; the next section discusses these factors. Moreover, 
as Chap. 4 indicates, the model served to define the dependent variables 

^ 4 ' . 

Vol&mes I, III, and IV discuss an earlier formulation of the local 
process of change. Volume VJ.U explores implications of the model. 
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(project effect and continuation) in operational terms, and to specify 
the data analysis scheme, used in the statistical analysis of the ef- 
fects of the factors. ' , 

The second premise of our approach — that institutionalization 
occurs in distinct ways for the individual teacher and for the school 
system — follows from the organizational structure and function of 
school districts. What may be a trial project from the standpoint of 
a district^ official can be a different matter to the teacher who imple- 
ments it.' The very act of putting an Innovation into practice can 
constitute a learning experience that produces an enduring change in 
a teacher's pattern of behavior or expands a teacher's use. of materials. 
In other words, the teacher institutionalizes some personal adaptation 
of the project's methods or materials; we use the term assimilation 
for this form of individual institutionalization.. 

Unless teachers assimilate its methods, a continued project can- 
not b^ fully institutionalized. But another condition also must be' 
net: The project-related change must be integrated into the normal 
operations of the school system. Unless the. project sheds its "special" 
status, it will face constant challenges. For example, 'its budget may 
be trinnned or eliminated. Or, project staff may turn over without new 
staff being assigned to or trained for the project. Or, principals and 
project directors may depart, leaving "leadership problems behind them. 
Or, central support activities may be denied to project ^participants. 
A project is institutionalized, then, only when the district's budget 
process, personnel allocation support activities, instructional pro- 
gram, and^f acili ties assignment routinely provide for the project's 
maintenance. We use the term incorporation for this form of system 
institutionalization. 

In summary, project-related changes ai^ institutionalized only a 
insofar as teachers assimilate them arzd the district ' s standard oper- 
ating procedures incorporate them. We have taken pains to ,make. these 
distinctions, for two-reasons. First, the two processes — assimilation 
and incorporation — have different dynamics that are influenced by dif-? 
f.erent factors: Assimilation can b©. described by a model .of individual 
learning; incorporation should be analyzed as an organizational process 
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composed of many decisions made by many actors. Second, assimilation 
can occur without incorporation, and vice versa. This "loose-coupling** 
between the necessary components of Institutionalization means that 
Innovations can be continued without being institutionalized. 

With these theoretical conceptions in mlnd# we collected data 
about project effects at both the classroom level and the school' system 
level. Chapter 5 presents an analysis of the factors affecting project 
continuation at the classroom level (i.e., assimilation). Chapter 6 
examines continuation at the district level (i.e., incorporation) and 
explores how the relationship between the two levels affected institu- 
tionalization. 

IDENTIFYING FACTQ'RS AFFECTING CONTINUATION ^ 

Our earlier research on Kow school districts attempt to make 
innovations work taught us an obvious, though often ignored, lesson: 
No single, simple, or sure .way can be found to effect. change . Nor is 
any single factor the key to success, whether it be iponey , new tech- 
niques, new superintendents, or new materials. Rather, the local 
process of change consists of the sum total of many everyday decisions 
made by many people dealing with the peculiarities of. their situations. 
Consequently, we collected data on a variety of factors and explored — 
by means of statistical analyses and field observations — how much 
weight each had in determining effective implementation ^and continua- 
tion. This section explains How the theoretical framework iliscussed 
in the preceding section^^helped to identify the major factors affecting 
innovation. . 

Figure 2.1 suggests that, an innovation, becomes a part of the on- 
going operation of the school district, which in turn is embedded in 
its coininunity and environmental setting. The nature of the school 
district, the connnunity, and the environmental forces have pervasive 
effects on why a change project is begun and what happens to it. Some 
of these influences are measurable and can be analyzed systematically; 
others are either not measurable in any practical and reliable way or 
are so idiosyncratic as to make comparisons from district. 'to district 
meaningless. 
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. Figure 2.] also indicates that three classes of factors directly 
affect the extent to which an innovation causes a change in the school 
district's delivery of educational services: 

o Federal and state policy 

o Characteristics of the innovative project ' 
■ ^ o The institutional characteristics of the local school district 

Conceivably, federal policies could influence the course of inuo- 
yations at. many different points, but most policies do no't\ Whether 
by design or by happenstance, their levera^i.=2 is generally limited to 
the beginning of pt.>jeci:s, as Fig. 2.1 illustrates by the arrow con- 
necting the federal fnput to the project's initiation. Specifically, 
the Title III program explicitly assumed that the sheer provision of 
"seed money" could help districts to implement and sustain innovations. 
^ We shall be examining in Vox. VIT.I the possibility of directing federal 
policies toward other aspects of the^^prcc^sc. 

VJhether the project 's beginnings were stimulated by federal policy 
or generated primarily by district concerns, innovations go through a 
complex interplay between project and institutional characteristics. 
The preceding isection identified three phases of that process — 
mobilization, implementation, and institutionalization — and pointed 
out that what happens during the first two. affects what happens in the 
third. Therefore, to identify factors "affecting con^tinuation (which 
is a product of institutionalization), we examined the "products" of 
the earlier phases. ' . . 

Mobilization : ^ 

The primary tasks in mobi lization ^are planning-related activities— 
e.g., defining a problem or need , seekinjg "solutions" ip terms of new 
educational methods . or approaches , locating funding opportunities, 
formulating a proposal to do a project,' and generating local support 
for the proposal. The results can be regarded as two 'broad "products": 
an adopted project and institutional atti'tudes toward it. 

The adopted project consists of ' ai series of more or less specific 
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decisions on what is to be, done r.nd how to do it. Project designers 
usually have an idea about an innovation that they saw, heard, or read 
about and now wish to implement in their district, school, or classroom. 
Whether one person or many participate in the design, the resulting 
project is usually a synthesis of a variety of ideas (often including 
current educational fads). This synthesis typically bears but slight 
re-semblance to its progenitor or any other project. Characteristically, 
then, the adopted project is an amalgam of innovative ideas and instruc- 
tional goals that we call the educational method or treatment of the 
.project. 

' The local choice of an educational treatment incorporates district 
and educational aims or objectives; it also assumes, often implicitly, 
expectations about the staff — e.g., what new behavior will be required 
of them, how and how much they will be expected to change, and so on. 
We call these new staff requirements the soope and complexity of- the 
change effort. 

The designers must also make a set of decisions about how to imple- 
ment the innovation, which we call implementation strategies. They in- 
clude implicit or explicit ^ choices about allocation of resources, 
selection of s.ites, staff, target group, project governance, involve- 
ment of parents, staff development, selection and development of mate-- 
rials, and evaluation procedures. 

In summary, mobilization encompasses an educational treatment, a 
scope of proposed change, and an implementation strategy. Phase I of 
the research suggested that the choice of. an educational treatment in 
itself has only ^ limited effect on project outcomes because similar 
innovations can be implemented in different ways with varying effective- 
ness. However, the scope of the project and the implementation strate- 
gies chosen had major effects on what happaned. Accordingly, Phase II 
collected data on these three project characteristics not only to re- 
affirm the earlier findings but to see whether similar results hold for 
continuation of projects after the end of federal funding. 

Mobilization engenders another consequence, often overlooked. The 
central office staff, the principal, and teachers develop attitudes of 
enthusiasm, support, indifference, or opportunism toward the coming 



42 



-23- 



innovation, which seem to be crucial for the fate of innovative efforts. 
These attitudes establish the "legitimacy" of the project, define the 
• priority accorded if. in the district, and determine people's commitment 
to the project. 

Broadly speaking, we found that these at titudes conformed "to ohe 
or the other of two patterns: opportunism and problem-solving. The 
"opportunists" seemed to be motivated primarily by the desire to reap 
the windfall of federal funds. Our Phase I research concluded that 
their projects were poorly implemented as a consequence. An early task 
of the Phase II research was to collect data on the status of 
"opportunistic" projects after federal funding ended. We suspected 
they would not be continued, and our suspicions were borne out (see 
Chaps . 3 and 6) . 

The "problem-solvers" took the projects much more seriously; in 
these cases, school districts appeared to welcome the help of federal 
funds in dealing with central educational concerns. The projects they 
adopted were more iikely to be implemented effectively because they 
enjoyed institutional support and a high level of teacher commitment. 
Because we hypothesized that institutional support also affects con- 
tinuation, Phase II used field interviews and survey questionnaires to 
ascertain the initial, attitudes of central offices, principals, and 
teachers. As will be discussed below, we found that institutional 
support did Improve the effectiveness of implementation and was a strong 
determinant of continuation. We also found that some locally chosen 
strategies were better than others for generating such support. 

Implementation - 

Our previous research suggested that implementation involves the 
adaptation of the project to its setting and vice versa. We. reported, 
in Vols.. II and III, on the difficulties that arise during this adaptive 
process and explained how school districts either overcame these' prob- 
lems or failed to adapt. 

Phase II focused on two additional issues: First, how dees the 
way the project was implemented affect its continuation? Because 
projects typically mutate during implementation, it is useful to 
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distinguish the original form of the project from its form three to 
■five years later; we call ^he latter the implementpA pr*nj^r^t. To .get 
a sense of its characteristics, we collected data on such factors as 
the extent of teacher participation in implementation decisions, the 
principal's attitude toward the project, and the percentage of project 
goals realized. 

The second issue we examined concentrated on implementation in 
and of itself. Rather than considering the implemented project and 
its effect as an ''input" to continuation, we also viewed them as 
"outcomes" in their own right. Our key questions for analysis there- 
fore were: Do the factors that influence implementation and the 
project's effect on teachers and students have similar effects on con- 
tinuation? Do different factors play different roles in achieving a 
"better" project as compared with sustaining the project and, if so, 
are th^re policy tradeoffs to be concerned with? 

We were well aware that people at the central office, school, and 
clasfirobm levels played distinct roles. For example, central office 
people can be heavily involved in mobilization when political and 
bureaucratic choices must be made, but they tend not to deal in tl-e 
day-to-day adaptation decisions characteristic of implementation. To 
det ermine how the involvement of central office staff members affected 
continuation, we collected data about their roles, attitudes and attri- 
butes before, during, and after implementation. We gathered similar 
data about principals and teachers. In particular, at the school and 
classroom level, we collected information about three types of variables 
the school's nvacmizationat :*lirnatp. ^ the school's cLmogvaphir ^iharaoter- 
7*.9/;C^v7, and tf-^.r^^^hnr* c^harci^tK' vis tics . 

Institutionalization 

Institutionalization can begin before the close of federal funding 
and ends long after it. The original staff can assimilate the innova- 
tion's methods during the first few years of the project's implementa- 
tion, or assimilation may occur more slowly, particularly for complex 
projects subject to staff turnover. Similarly, the project may be in- 
corporated at the outset into the district's procedures ^viz. , those 
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procedures involving budget, personnel, staff support, instruction, 
and facilities). For example, initial plans /jbput alternative schools, 
particularly those involving desegregation , may require facility con- 
st^ruction, suppprt activities, and personnel assignments that become 
institutionalized as- they are implemented. But. this , case is rare, as 
Chap. 6 argues. Typically, district officials postpone decisions about 
the fate of. a project until that stage of the budget cycle when the 
"soft" money from the federal grant runs out. V7e belieye this "formal" 
district-level decision resembles the original decision to adopt the 
project (with the difference that "adoption" this time entails the use 
of district funds). For example, the decision to continue a, project, 
like the decision to' adopt, is based on loc^l bureaucratic and political 
considerations as well as assessments of the educational value of the 
project. Chapter 6 analyzes the factors affecting this 'decision. 

Phase II also assumed, that a project's passage between continuation 
and institutionalization sets in motion yet another cycle of mobiliza- 
tion and implementation. Figure 2.1 depicts this cyclical process by 
showing an arrow going from institutionalization back to mobilization. 
In other words, pur data collection and analysis did not equate continua- 
tion with institutionalization, just as adopting a project is not the 
same as implementing it. 

Phase I found that the implementation strategies chosen criti- 
cally determined the cours^ and outcome of implementation. Accordingly, 
we examined the strategies selected by local district officials at the" 
continuation stage, to see if they were similarly powerful in determining, 
the extent of institutionalization. Based on Interviews with school 
district officials. Chap. 6 suggests that some strategies are more 
effective than others for that purpose. 

Summary 

In summary, institutionalization is the process by which a contin- 
ued project gains permanence. The extent to which a continued project 
is assimilated by teachers and is incorporated into the district's 
standard operating procedures depends on a variety of institutional 
and project factors that are inputs to or products of the prior local 
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Table 2.1 

FACTORS AFFECTING PROJECT CONTINUATION 
IN THE CLASSROOM 

Federal input: project funding 

Project characteristics 

Educational fnethod or treatment 
Scope of proposed change 
Implementation strategies 

Institutional setting ' 
Organizational climate 
School characteristics 
Teacher characteristics 

history of the innovation. We therefore gathered information on two 
types o^ variables. First, data were collected on variables reflecting 
project design choices and characteristics of the institutional setting, 
which we hereafter call factovs or independent variables, because they 
can be affected- by policy or can.be taken into account in. planning. 
Table 2.1 lists the fact ors studied in Chap. 5 s analysis of continua- 
tion in the classroom. Second, data were collected on project "outcomes" 
that were interim to continuation — e.g., how effectively the project was 
implemented. These variables can be called dependent variables because 
they are the product of the interplay among the project and institutional 
factors. Chapter 4 presents our assumptions. about how the factors and 
dependent variables are related at the classroom level, and also dis- 
cusses our operational measurements . of the classroom level dependent 
variables. Chapter 6 provides similar information about district-level 
independent and dependent variables. 
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Chapter 3 

DATA COLLECTION AND THE NATURE OF THE PROJECTS 



The basic data collection plan of Phase II called for revisiting 
and resurveying many of the 194 Title III projects analyzed in Phase I. 
Our goal was to conduct a survey of 100 former Title III projects one 
to two years after the end of federal funding Using a telephone survey 
to supplement the extensive dat-a base gathered about these projects dur- 
ing their last or next-to-last year of federal support, we chose a 
sample that would provide the maximum amount of information about con- 
tinuation. Our model of the factors affecting the process of change 
called for analyses at both the classroom le\^el and the district level. 
Accordingly, the survey consisted of • separate questionnaires adminis- 
tered to teachers, principals, and superintendents. To help in design- 
ing the questionnaires, and to explore aspects not amenable to quantita- 
tive analysis, we visited 18 of the districts and conducted personal 
interyiews with /project participants and school officials. . 

This -chapter discusses the data collection activities in more de- 
tail, and then describes some characteristics of the projects in our 
sample. 

SAMPLE SELECTION FOR PHASE II 

The sample for Phase II was drawn primarily from projects surveyed 
in Phase I, which had collected data in two ways: 

1. A nationwide (18-state) survey of a sample of 293 projects 
in their last or next-to-last year of federal funding, 194 
^ of which were sponsored by Title III of ESEA. For each 
project, we interviewed an average of nine members of the 
school district (the superintendent, the federal program 
manager, the project director, two principals, and four 
teachers). We mostly used closed-ended . questionnaires that • 
explored a wide variety of topics but focused on project 
initiation and implementation. 

47 . 



-28- 



2. Field titudies of 29 projects from a survey sample. These field 
studies, consisting of observations in classrooms and inter- 
views with a wide variety of project parti'dpants and LEA 
officials, focused on the local process of change. 

This extensive data base provided a unique opportunity to select a 
sample of innovations that we already knew a good deal about. Accord- 
ing to an agreement .with USOE, we had sufficient resources to investi- 
gate 100 of thesc: 194 projects. We could have selected a random sample 
of the Phase I proj ec ts , which would have inc luded a large number of 
discontinued projects, but since the study's primary concern was to > 
explore how policy might influence the continuation and spread of in- 
novations, it was more appropriate to choose projects, for survey analy- 
sis that were actually continued, at least to some degree.^ 

Therefore, to reduce the sample to 100 former Title III projects, 
we conducted a telephone survey of the 194 eligible Title 111 projects 
originally surveyed. Using a largely closed-ended questionnaire, we 
asked school district officials in each of the sites to describe what 
had happened to the project, whether it was being continued, and, if . 
so, which schools were now uising the project in any form. 

The responses revealed that about 30 percent to 40 percent of the 
projects in the original sample had been completely discontinued by the 
district; former project schools were not using the project's methods 
or. materials one or two years after the end of federal funding, and 
we sometimes had considerable difficulty even locating former project 
participants. Checking back on the original suTrvey information about 
these projects, as- well as using telephone reports from former projec^t 
participants, we found a clear reason for their disappearance: The 
projects were not implemented effectively, and in many cases were hardly 
implemented at all. Participants reported that implementation was 
either extremel)? dif f icult or was merely symbolic because the project's 
goals were peripheral to their educational concerns. In short, most 
of these projects were probably "add-ons,*' adopted more to receive 
federal funds than to solve pressing educational problems. Chapter 6 
examines . these and other reasons for discontinuation at the district 
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level. It uses the data collected in the telephone survey and in . 
Phase I to compare the attributes of discontinued and continued proj- 
ects. In the Phase II survey, we collected no further data about these 
discontinued projects for several reasons. First, as far as we could 
tell from selected pretest visits, from the telephone survey, and from 
Phase I data, most teachers in these projects were no longer using 
project methods or materials in their classrooms. Second, we had 
collected enough data in Phase I about the limited clas&room implementa- 
. tion of these projects to explain their discontinuance. We accordingly 
decided to include in the Phase II survey only projects that were con- 
tinued and appeared to have had direct effects at the classroom level. 
,In summary, we studred district discontinuation using information col- 
lected during Phase I,, and studied continuation at both district and 
classroom levels using newly collected data from Phase II, 

To construct the Phase II survey sample, the original sample of 
194 former Title III projects was narrowed down to 95 candidate proj- 
ects, each of which still existed, in some fashion, in its district. 
To reach our design' goal of analyzing 100. proj ects , we needed to add 
20 projects, for a total sample pool of 115. (The extra 15 projects 
were necessary to compensate for an exfiected district refusal rate of 
about 13 percent.) We selected 20 former Title III projects not in 
the original sample, using information 'derived from project materials 
and telephone interviews with state and local education officials. 
Table 3.1 shows the distribution of the 100 projects across 18 states 
where Rand's research was conducted. 

The original group of change agent projects was not a probabilis- 
tically representative sample of all federally sponsored projects, but 
rather was a purposive sample that covered a broad range of innovations 



• We know of no teachers who went on using. the project in these 
cases of total discontinuation, since we did not choose to spend our 
limited research budget to verify classroom discontinuation beyond the 
procedures described above; it is possible that these projects were 
being used in the classroom to some extent. This possible underesti- 
mate of classroom continuation introduces a potential for bias of an 
unknown degree into Chap. 5*s classroom level analysis. 
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Table 3.1 



NUMBER OF SAMPLE PROJECTS BY STATE 



Number of Number of \ 
State Projects ^Schools \ 

\ 



California . 


20 


36 


Colorado 


3 


5 


Connecticut 


7 


11 


Florida 


k 


6 


Georgia 


k 


8 


Illinois 


10 


20 


Kentucky 


1 


2 


Massachusetts 


3 


6 


Michigan 


■ 4 


5 


Missouri 




8 


New Jersey 


10 


14 


New York 


5 


9 


North Carolina 


15 


24 


Ohio 


3 


8, 


Pennsylvania 


2 


4 


Texas 


2 


4 


Washington 


3 


6 


Total 


■ 100 . 


176 


NOTE: The s 


amp ling pool 


also 



, included two projects from Arkansas 
and two from Pennsylvania. 

in a wide variety of state and local settings. Consequently, the proj- 
ects analyzed in. Phase II do not constitute a statistically representa- 
tive sample of all past Title III projects. Indeed, because we did not 
survey projects that were totally discontinued, the Phase II survey 
sample undoubtedly contained a' higher proportion of "successful" innova- 
tions than would a probabilistically representative sample. Nonetheless, 
even among continued projects, one would expect a wide variation in the 
nature and extent of their continuation . Such variation did occur in 
our sample, as subsequent analysis will show, and thus we coul.d examine 
what happened to many different types of innovations in many different 
local and state settings.. 

Given the purposive nature of the Phase II survey sample, general- 
izability of the findings is subject to question.- Moreover, the method 
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of sample selection may have introduced errors into the analysis that 
would bias the results. Though these metliodo logical difficulties must 
qualify any specific finding, they are not as severe as to invalidate 
the overall import of our exploratory analysis of what promotes con- 
tinuation and why. 

FIELD RESEARCH AND SURVEY DESIGN 

The Rand staff visited 18 sites before or after the administration 
of survey questionnaires during January 1976. The purpose of the prior 
visits (during the spring and fall of 1975) was to formulate a concep- 
tual framework of continuation as well as to conduct survey instruments 
Following two waves of pretests (conducted by Response Analysis Corpora 
tion and Rand during fall 1975) and the administration, of the survey 
by Response Analysis Corporation, the Rand staff visited additional 

sites in the spring of 1976 to explore areas not adequately covered by 

2 

the surveys (e.g., school district decisionmaking). Most '^r on- 

site visits (which .averaged about 3 person-days per project; Wic-re uo 
locations that had been previously visited — often several times — during 
Phase I. Our detailed familiarity with these districts, their staff 
relationships, and project charac teristics gteatly facilitated our re- 
search activities: classroom observation, group and individual discus- 
sions with project participants and community people, and interviews 
with LEA officials. We believe that such sustained contact with the . 
realities of innovations — their problems as well as their triumphs — 
is necessary to understand the process of change and to interpret sta- 
tistical analyses of factors affecting the process. 

2 ' 
To supplement this field research, the Rand staff also visited 

several projects that were using USOE's Project Information Packages, 

exemplary innovations being disseminated by USOE, and visited several 

innovations that were started and funded solely by LEAs . This research 

gave us a basis for comparing different approaches to initiating and 

supporting innovations. Our strong impression is that, despite some 

noteworthy differences, all these types of innovations face similar 

difficulties, go through a similar process of change, and are affected 

by similar local factors. To examine, strategic questions of sustaining 

innovations, meetings were also held at Rand with various LEA officials. 
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The survey, itself questioned three classes of respondents — super- 
intendents, principals, and teachers — for whom we designed three dif- 
ferent survey instruments. The superintendent *s ^questionnaire con- 
tained a number of open-ended questions and was administered as a 
personal interview that lasted between twenty and forty minutes. It 
focused oil district decisions concerning the beginning and the continua- 
tion of the project and on general information about the district,. The 
superintendent also was asked to evaluate the principals of one or two 

schools in our sample. A total of 100. superintendent questionnaires 
3 

were administered. 

We also interviewed 171 principals f5r an average of about forty 
minutes. . Their questionnaires elici ted in format ion on the project's 
design, the school's early involvement in the innovation, staff atti-<^ 
tudes toward the proje^^t, and the project's effect on the school. 
Principals were also asked to evaluate a sample of as many as eight 
project teachers.^ 

From the 171 schools, 1072 teachers . f i lied out self-administered 
questionnaires that. covered a wide variety of topics, including the 
nature of the project, its implementation, its effect on teachers and 
students, the nature and extent of project continuation, and the school's 
organizational climate.^ They were also asked to take a short verbal * 
ability test (94 percent of the sample did so). 



In 12 percent of the cases, a high district official was inter-- 
viewed instead of the superintendent because of (a) refusal (2 cases), 
(b) size of the district, which meant that the superintendent was un- 
likely to be familiar with the project (7 cases), or (c) the superin- 
tendent's recent appointment to the district (3 cases). 
4 

A second telephone survey in 'the fall of 1975 gave us exact in- 
formation on the schools currently using the project in some form and . 
on how long the principals currently at the schools had been there. 
Both current principals, and to a limited ex ten t , former principals 
were interviewed. Insofar as possible, two -project schools from each 
district were eligible for the- s^nmple; in the case where a district had 
more than two schools that had had the project for at least two years, 

a random sampling procedure was used to select the schools . 

5 a 

In those cases where more than eight teachers were eligible in a 
school, a random sampling procedure was used to select the teachers 
rated . 
6 

As many as eight teachers, who had two years experience with 
the project while it was federally funded were asked t^^ fill out 
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In s.viinniary, we revisited and resurveyed Title III projects one 
to two years after the end 6f 'federal funding. Using-our existing 
data base and 'a telephone survey, we selected a nationwide sample of 
projects and respondents comprising: 

-^-1. 100 former Title III projects that were "continued," to some 
extent, after the end oV federal fundihg. 
2. - IDO superintendents or key LEA officials,- 171 principals at 
schools where the projects had reportedly been, con t inued , and 
1072 project teachers. 

Subsequent sections will discuss survey questions and respondents' 
answers. 

THE P ROJECTS IN THE PHASE II SAMPLE • 

The original sample of 194 projects included about 45 not intended 
for the classroom, for example, scho^ol or district administrative changes 
such as Management-by-Objectives, and enrichment, activi ties such as drug 
education programs. We eliminated these nonclas'sroom projects whether 
they were continued. or not (most of them were dropped immediately at the 
end of federal funding), and focused exclusively on classroom innova- 
tions that had been more or less seriously tried. 

These innovations were extraordinarily varied. They differed in 
the grade levels they covered— 71 percent of project teachers were in 
elementary schools, and 29 percent in^secondary schools ; ^ thei r target 

questionnaires by RAC interviewers (the retuTn rate was 97 percent) in each 
project school; if more than eight teiachers were eligible, a random sam- 
pling procedure was used. However, in four schools having a large number 
of project teachers, all e?ligible teachers were given thjsl questionnaires. 

7 ■ ^ ^ • 

Because school systems across the country differ-in th^ way they 

def ine the grade— level composition of their schools, the division be— 
tween elementary, junior or middle schools, and senior high schools is 
not comparable nationwide. The exact distribution of projects accord- 
ing to the grade levels they covered in ojiir sample of 171 schools was:. 

Primary grades only , 25% 

Upper elementary grades only . 8 

Both primary and elem. grades ....... 38 - 

Secondary grades .> 24 

Both elementary and secondary 5 
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gfoups — e.g., 19 percent of the projects were aimed at students whose 
.achievement was below mos t- students in their school; the selection of 
the school site and the staff-^ — e.g., 16'percent of the projects involve'd 
all schoals .in the disJ'trict and 38 percent of the schools had all their 
teachers participating; the size of the projects — e.g., the number of 
project schools varied from one site to 44 sites, with an average of 
five project schools in a district, and the number of students per 
project averaged about^ 300 pupils with some projects being single class- 
rooms of 20 students and others involving all the students' in a school 
(as many as 1150); the .characteristics of the host school district — 
e.g., 38 perdent of^^the projects were in cities, 30 percent in suburban 
areas, il percent in towns, arrS 2i percent in rural areas.' The largest 
city school district had an average daily enrollment of almost 750,000, 
and the smallest rural school district had 385 pupils. 

Perhaps the most interesting variation was in the substancCc of the 
innoyations. At the most general level, the projects were very simij.ar; 
all focused on classroom-related problems and can be seen as serious 
attempts to change student performance. To achieve' this common purpose, 
however, local school personnel identified a broad variety of project 
striitegies. Some projects comprised highly specific techniques that - 
focused narrowly on student achievement. For example, an- individually 
prescribed elementary school instruction project pretested elementary 
students and .then identified materials that could^ help them master items 
they had missed on the pretest. A similarly designed mastery learning 
project used a set of behavioral obj ec t ives in combination with coded 
teaching materials — games, books, TV tapes, and so on. Other projects, 
sought to improve student performance by focusing on students' self- 
concept and school-related attitudes. One teacher described the latter 
kind of project as "an attempt to modify students* attitudes toward the 
school, community, the staff and peers by concentrating on a values-rich 
curriculum. " 

Some projects took a more indirect and general approach. For ex- 
ample, some projects concentrated on general staff development and 
training activities not explicitly tied to the classroom. One such 
project offered training in diagnostic and prescriptive methods; it 
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consittted of study and discussions with teachers over thirty school- 
months, with weekly three-hour mee-zings. Another project focused on 
staff training;, it was "designed to make teachers and administrators 
more aware of the feelings of children and teachers. Evening seminars 
were offered^for interested teachers and administrators in which the 
group was introduced to methods of effective education and practiced 
these techniques with each other." Other projects elected to involve 
parents as an indirect strategy' to change student performance. One • 
project offered both Parent Effectiveness Training and Teacher Effec- 

n 

tiveness Training; the teachers received video-taped self-evaluation 
and inservice training on effective parent conferencing. Another proj- 
ect cmph asized i>ome— school relations and hired aides as home— school 
coordinators. 

Yet another group of projects aimed for comprehensive change in 
classroom practice, with the explicit goal of improving student atti- 
tudes and achievement. These projects were much less specific than 
the more narrowly focused achievement projects (such as mastery learn- 
ing) and made project training an integral part of the changes being 
implemented in the classroom. The following teacher, description is 
representative of^'projects of this nature: 

The program was designed to. aid teachers in the process of 
individualizing instruction in the areas of language arts, 
math, social studies and science. The program focused on 
enhancing student academic growth as well as their self- 
concept and feelings about school. In order to accomplish 
individualization in these areas, classrooms were reorgan- 
ized and teachers developed learning centers in their rooms 
for each subject area. Students were taught to use new 
instructional materials many of which were developed by 
- project teachers ; activities varied depending on the level 
of the child. Teachers were given training in setting up 
these new classroom procedures and materials and in teach- 
ing to objectives designed for the child's level of ability. 
A computer was used to assist the teacher in the area of 
recordkeeping and progress reports to the parents. 

All the projects , then , addressed central educationa.1 needs of the 
students in one way or another. None were concerned with ancillary or 
"general enrichment" activities. Some projects focused directly on 

' ■ c;r: ■ 
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, specified academic deficits or on student attitudes and motivation 
related, to achievement. Other projects attempted to Improve student 
performance by changing the behavior and supplementing . the skills of 
teachers and parents. Still other projects were concerned with student 
performance in both cognitive and affective areas, striving for com- ^ 
prehensive change in classroom organization, climate, and activities. 

All this variation was not accidental. The ot Lginal sample was 
deliberately chosen to span a broad ranjge of projects in many differ- 
ent institutional settings. This approach enabled us to collect in- 
formation about innovations that clearly differed in their outcomes 
and in factors affecting the outcomes. For example, some projects were 
conducted at several schools in a district, ethers at only one school; 
we therefore were able to analyze, using statistical methods, the 
effect that the number of project schools had on continuation. In 
short, variety in project characteristics and institutional settings 
allowed us to general izer^-wi thin limits — about the factors affecting 
project impact and continuation. 

Generalization exacts a price, however.. For example, because the 
Innovations varied markedly in the substance of their educational method,' 
we were able to explore whether some types of educational treatments were 
consistently implemented more effectively -than others and more likely to 
be continued. The price exacted, however, lies in the precision of our 
generalizations. We would like to, but cannot, make statements like 

o 

"Reading Program A is better than Reading Program B." Instead, we had 
to group the innovations in our sample according to the types of educa- 
tional method they represented and, consequently, limit our generalizations 
to Inferences about these abstract types. ' 

The educational method or substance of an innovation generally con- 
sists of an amalgam of instructional techniques pieced together to fit 
the Intervists and concerns of- the project's designers. Goals, means, 
and strategies are all mixed together. The product, as specified in 
proposals or more refined plans, tends to be an innovation virtually 

8 

Although we have data with which He could make at least reasonable • 
evaluative judgments on each project, we decided to forgo doing so for 
the sake of formulating general principles that would be applicable to 
most projects. 
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unique to its district. To group these ''unique" Jnnovatlons into com- 
parable types of educational methods, we collected data from project 
teachers about the rcinge of educational techniques used on the projects 
and devised an analytical classification using statistical methods. 

The survey asked teachers to check off the educational techniques 
employed in their project. Table 3.2 enumerates the educational tech- 
niques and lists tlie number of times each technique was mentioned.- 
Considering the lack of clarity and diffuseness of educational treat- 
ments, it is not surprising that teachers never characterized their 
project in terms of a single technique. It is fortunate that they. 

Table 3.2 
.EDUCATIONAL METHODS -OF PROJECTS 



Speaial projects may use a variety of different 

techniques. For the techrdques listed below ^ 

please circle the letter preceding each technique Number. 

you used as part of the project in this school ' of Times 

during the petHod of initial federal funding. Mentioned 

Individualized instruction 700 

Behavioral objectives 641 

Criterion-referenced testing 335 

Diagnostic and prescriptive methods 554 

Differentiated staffing in elementary schools ...... 132 

Paraprofessional staff (teacher aides, etc.) 495 

Team teaching 212 

Instructional specialists in addition to teachers . . 364 

Counseling and' guidance specialist's 245 

Multi-age grouping (at least 3 grades together) .... 115 

Nongraded student evaluation procedures^ 272 

Open education 192 

Learning centers . , /493 

Cross-age tutoring 2O6 

Performance incentives for students 296 

Teacher-student contracts J 349 

New curriculum .. 343 

Educational technology (computers, etc.) 133 

Development of new materials 550 

. New management techniques at the school level 254 

Field trips ,. . 30O 

Parent involvement in. the classroom 254 

Use of community resources 284 
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did not. It would be confusing and almos • meaningless to ideriify a 
project as a ''parent involvement project" without specifying what com- 
bination of techniques was used in conjunction with parent involvement. 
The data, when analyzed by correlations among the various techniques, 
clearly suggest t:hat certain techniques went together in practice. 
Moreover, , it seems reasonable to hypothesize that likely combinations 
of techniques were united by underlying and more general educational 
methods- Since there were relatively few likely combinations of tech- 
niques, we suspect that these general, educational methods were also few 
in number . ,. " 

These theoretical premises suggest the use of factor analysis, 
a statistical procedure designed to reduce a large number of elements, 
such as diverse educational techniques, into a smailer number of under- 
lying dimensions. The validity of factor analysis, apart from teciinical 
considerations, rests on whether the factors, or underlying dimensions, 
have any valid interpretation or meaning. Our factor analysis of the 
data of Table 3.2 indicates that the interpretations of the factors were 
meaningful and provided an efficient means for analytically describing 
the educational methods of innovative projects in the Rand sample.^ 

Table 3.3 displays the results of a factor analysis designed to 
discriminate a small number pf underlying or general educational methods. 
Five analytical types of educational methods emerged: 

o T>y:Hi)iduaLiy.atioyi techniques^ innovations involving student- 
centered procedures, methods, and materials for dealing with 
the learning needs of each pupil individually; specific tech- 
niques include criterion reference testing and diagnostic- 
prescriptive methods. 

o Classroom organization change^ projects including teacher-. 
, centered innovations designed to alter the usual pattern o-f 



9 ' 

This report does not discuss the statistical details of the factor 
analysis. The factor analysis enables a comparison among the educational 
methods of the diverse and idiosyncratic projects based on . similarities 
among the projects. However, this statistical procedure cannot make 
"apples out of oranges." Therefore, subsequent analysis controls for 
some other project features that cannot be characterized by the factor 
analysis . 
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Tablo 3.3 

FACTOR ANALYSIS OF KDUCATIONAL METHODS 



Educational Tuchniijue 



.Individualized Instruction 
•Behavioral objectives 
Cr 1 ter Ion-referenced 

testing 
Diagnostic and prescriptive 

methods 
Differentiated staffing in 

elementary schools 
Paraprof ess lonal staff 

(teacher aides, etc.) 
Team teaching 
Instructional specialists 
in addition to teachers 
Counseling and guidance 

specialists 
Multi-age grouping (at 

least 3 grades together) 
N'jngraded student evalua-* 

t'ion procedures 
Open education 
• Learning centers 
Cross-age tutoring 
Performance incentives for 

students 
Teacher-student contracts 
New curriculum 
Educational technology 

(computers, etc.) 
Development of new 

materials r 
New management techniques 

at the school level 
Field trips 

Parent involvement in the 

c lassroom 
Use of comraunit • re^jources 



Sum 



squares 



Rotated Factor Loadings 



(.'onnunity 
Invii ', vumunl 



.'^01 
. l-iO 

.006 

-.0 3^ 

.062 

.^2 5 .. 
.21*7 

.196 

.132 

. 228 

.093 



.^36 



. 33^ 

.059 
.231 
.117 

. 1 32 

. 361 

.024 



. 706 
.758 



Indlvldual- 
Izat ion 



2.857 



.681 
.451 



.564 
. 760 



.09 7 

.428 
.09 5 

-.00 7 

.048 

-.140 

. 195 
. 1 5n 
.414 
.227 

.381 
. 367 
.071 

. 190 

.273 

.401 
.020 

.215 
.057 



Specialists 



Classroom 
Organlzat ion 



Curriculum 
Revislcns 



-.200 
.135 

.088 

.029 

.286 

.138 
-.013 



.633 
.687 



-.116 

.048 ■ 
-.174 
-.0S8 

. 176 

[ .446 
.169 
.332 

-.037 

. 26 7 

.302 
.176 

.085 
.224 



1.715 



.154 
.095 

.Hi 

.089 

I .651 

.198 
I .668 

.090 

-.020 



.599 



.625 
.452 



.183 
; .450 

. 310 
.385 
.235 



.044 
-.036 



.281 
.148. 



.198 
.161 



2.628 



.035 
.104 

.375 

.138 

-.tJ8'4 

-.066 
-.085 

.171 

.050 

.34 4 

.213 
.062 
-.005 
. 164 

-.254 
-.123 



.578 
.642 
.407 



. 100 

.007 <; 



.002 
.030 



1 .419 



Coramona 1 tty 



.568 
.263 

.479 

.606 

.526 

.426 
.523 

.476 

.495 

.562 

.485 ' 
.454 
.497 
.42 3 

.507 
. 380 
.518 

.469 

.443 

. 341 
.606 

.591 
.654 



11 .295 



Loadings refer' to the correlation between an educational technique and a factor of general educational 
method. The boxed entries indicate high loadings on a factor that serve to define the meaning of the far- 
tor. Commonality and sum of squares are statistical nie.isurcs th:it indicate the technical validity of, the 
analysis. 



interaction betweer/ teacher and student (e.g., open class- ' 
•rooms) or teacher and teacher (e.g., team teaching and d i f- 
feren'tiated staffing). ' 
Curriculum revisions-, projects for developing new curricula 
or introducing educationaT techniques to supplement tho. 
curricula. 

Communitij involvement, projects employing community resources 
or parent participation. 

Speaialuibi) p^p student needs, projects that used specialists 
and guidance counselors to meet students* needs. 
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The names of these categories of methods were chosen to reflect the 

combinations of techniques revealed in Table 3.3 and to suggest gen- 

1 o 

eral methods widely recognized by educational specialists. 

The factors, or underlying educational methods, may be in terprcf '.^d 
in several ways. Each project in the sam'^le received a score on each 
factor. A factor score ran be thought of as representing the "amount" 
of a particular educational method used in a project. For example, a 
score of 2.0 on the classroom organization factor means that the proj- 
ect design attached great importance to classroom organization change 
activities. (Fac tor. scores are distributed as a standard normal dis- 
tribution. A score of 2.0 represents two standard deviations greater 
than mean project score.) If an innovation scored high on classroom 
. organization methods and scored low o.i the other factors, we would be 

justified in calling the project a classroom organiza t ion ijinova tion . "^"^ 

/' 

Any project in the Rand sample can be der^cribed similarly Wi terms of 
its mix of the. five educational methods. 

Our subsequent analysis uses this factor-score description. In 
Phase I we found that the educat ional methods of a project did not 
explain much ot the variation in project oi't; comes; Chap. 5 asks whether 
.the same result holds for continuation. We rdlso will examin/ the rela- 
tionship between the choice of educational method and o the r /proj ec t 
characteristics tha t migh t , a f ;:ec t continuation. For this chapter's 
purpose — to. describe the nature of projects in our sample — the most 
pertinent relationship is that between educational meth.od and the amount 
of federal funding. Projects that focused heavily on individualization 
were the most heavily financed; otherwise, funding and educational method 
were not strongly related. 



. 10 

The statistical validitiy of factor analysis depends on technical 
considerations not discussed here. In general, the factor analysis of 
Table 3.3 has relatively good but not very strong statistical character- 
l^-tics. Other statistical methods using nonparamet ric assumptions and 
clustering analysis yielded essentially similar factors. The fact that 
che categories defined by the factor analysis make intuitive sense gives 
us reason to trust its results. 

^^Ihe high-scoring projects on the various factors were checked 
agavnst fieldwork and other quali tat ive information and found to be 
generally consistent with our understanding of the substance of the 
project. 
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Wu did not find lb easy to measure the amount of money available 
to a project. Depending on such c i rcums tanees as the type of innova- 
tion or the .size and management style of the school district, it can 
be difficult to d is t ingui sh project funds from other_ monies aval lab le 
to a sc'ijol that support the project indirectly. Rather than trying 
to determine th<? true "costs," we" co I lected data (from official project 
records or answers to the Phase I survey) on the federal funding levels. 

In measuring funding level, we needed to take into account all 

incremental monies spent on the project. Although Title III grants were 

intended to cover most if not all of the project costs, Title III was 

often not the only source of funds. Mrny local projects also obtained 

funds from state categorical pro.'^rams , special LEA appropriations, or 

private foundations. Moreover, some LEAs "packaged" federal grants from 

such sources as Title I, Education Professi(5ns Development Act (EPDA) , 

or National Defense Education Act (NDEA) to supplement the primary Title 

in innovative project grant. Consequently, to examine the effect of 

fundlTig levels on project outcomes, we used the sum of the special grants 

from alL sources employed to support local Title III projects.. 

Even this choice did not resolve our operational problem. The funds 

were awarded over a three-year period. What would be the best measure 

of the resources available to a project? The total sum of all funds 

over the three years? The average funding per year? The largest yearly 

funding level? We chose the largest yearly project funding level for a. 
12 

number of reasons. First, a simple measure of the average grant re- 
ceived over several years could be deceptive because of significant year- 
to-y^ar variation. For example, it was not unusual for a Title III 
project to receive $125, 000. in support in the first year, $75,000 tihe 
second year,- and $85,000 irV^the third and final year. Second, prelim- 
inary analysis uncovered no discernible patterns in project funding 
histories, thereby precluding, generalization about trends in funding 
levels. Third., analysis of the effects associated with the largest 
single year of special project funding appeared particularly relevant 

12 

In Phase! we used each of these measures in preliminary analy- 
sis; the largest yearly funding level i^roved to. he the best measure. 
See Vol . II . 
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to tlK-v quGSLions this study addrosseM. There has been considerable 
debate in policy circle^s concerning the existence of an "optimal" level,- 
of outside support, ?f loca I projects 'are "to make a difference." Two 
little money, many say, simply means ine f^Wcilency . Conversely, it is 
often argued that federal support of expensive projects is wasteful 
because districts are unlikely to be able to continue them once federal 
funding ends. By examinin-^ the relationships between tiie largest annual 
funding figure and outcomes of Title III projects — where federal support 
is explicitly seen as "seed money"— it is possible to see whether the 
size of the "seed" makes any difference. 

The projects in our sample displayed a great range in their largest 

iinnual funding level — from $10,000 to $ 335 , 000— wi th most projects funded 

13 . 
at around $100,000. Although these amounts, which are absolute figures 

uncorrected for district size or budget, do not represent a high percent- 
age of a school district's annual expenditure, they vary sufficiently 
to allow us to explore whether the Ip.vel of incremental^ and uncommitted 
funds mattered for project effect and continuation. 

We defer discussion of other project characteristics to Cliaps^ 5 
and 6. 



The distribution of funding was skewed in a lognormal shape, with' 
most pr;.ojects funded at approximately $70,000 for their largest annual 
funding level. See Vol. II for a more detai led. discussion of this 
dist ribut ion . 
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CLAS SROOM DATA- ANALYSIS MODEL AND MEASUREM ENT 
OF DEPENDENT VARIABLES 

An innovarion may afCuct teachers, students, schools, and the 
distrit't in diverse ways and 'at different times in the innovation's 
Life cycle. The challenge for an exploratory analysis of the tyj^e 
Rand undertook was two-fold: to know what to measure as well as how 
to measure it, and to specify relationships among the various measure- 
ments. In short, we had to formulate a data analysis model. 

The model described in this section deals with clas,sroom-leveJ 
issues; Chap. 6 t rea ts dis t ric t-leve I issues. Tlie model was designed 
to generate empirically based hypotheses rather than test or estimate 
previously established relationships. Though the form of our statis- 
tical analysis resembles procedures used in hypothesis testing for 
nonexperimental data, it would be misleading to construe the . findings 
as anything more than tentative. Ouj: objective was not to establish 
hypotheses that ,could be formally tested, but to arrive at a deeper 
understanding of how to make innovaticii3 work. In sum, we designed 
our data analysis model to enable us to draw policy-relevant infer- 
ences and to arrive at better hypotheses (i.e., a better model) and 
more refined assumptions. 

This chapter describes the model , operational measurements of 
the dependent variables, and limitations of the analysis. 

DATA ANALYSIS MODEL OF PROJECT EFFECT AND CONTINUATION 

We posited a simple, statistically tractable ,, causal model of 
how the variables suggested by our theoretical conception of the local 
process of change affect each other. Namely, we assumed that: 

o An innovation can have four types of "outcomes" at the class- 
room level: an implemented project, changes in teaching 
style or behavior, changes in student performance, and con- 
tinuation of the project's methods or materials. 
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o The outcomes or depcndcMit variables can be ordered by the 
time sequence of their occurrence, which also corresponds 
to their cause-and-ef f e'ct relationships; some occur i)ri(^r • 
to others, and earlier variables can affect later ones, hut 
not, vice versa. Specifically, the star:istical analysis 
- assumes that implementation occurs first, followed by change 
in teachers, followed by change in students and by continua- 
tion of the project's methods or materials. 
:) All the factors or independent variables identified in Chap. 
2 can affect any of the dependent variables. 

Figure 4.1 is a schematic diagram of this theoretical "causal" model, 
which is called a recursive model. ^ 

The model comprises the central components of statistical analy- 
ses,, presented in Chap. 5, that are designed to determine the extent 
to which each factor (or independent variable) leads to better c lass- 
room outcomes — i.e., to effective implementation ,. teacher change, im- 
proved student performance, and project continuation. The. approach 
is to present statistics that (a) "estimate" the weigh t each . fac to r 
has in explaining variation in project outcomes, and (b) indicate the 



The recursive model can be stated algebraically as: 

= f(X)'+ 

= f (Y^,Y2,X) + 

Y^ = f(Y^,Y2,X) + 

Y^ = f'(Y^,Y2,X) + ^5 

where X is a. vector of factors or independent variables, Yj^ is imple- 
mentation effectiveness, Y2 is teacher change, Y3 is student improve- 
ment, Y4 is continuation of project methods, Y5 is continuation of 
project materials, and is an error ternv. Because the discussion 
in the text aims at a general audience, we avoid using such technical 
terms as "exogenous" and "endogenous" variables to explain the recur- 
sive system. 



Facfors or Independent Variables 




Dependent Variables 


Federal input: funding level 

Project characteristics 
Educational method 
Scope of change 
Implementation strategies 

Institutional setting 

Organizational climcie 
School characteristif:5 
Teacher characteristics 


. Improvement in 

^ — ^ student achievement 
^^^^^^^^^^."""^ ^ Or behavior 

Implementation Teacher 




effectiveness change 

Continuation of 
— • — ^ project methods 
Or materials 



Fig. 4.1 — Classroom-level data analysis model 



statistiical significance of these estimates for the characteristics 
of c sample. In a formal sense, these statistics cannot be used to 
refute our model — i.e., to test whether some of the factors assumed 
by the model do not significantly affect project outcomes. Yet our 
policy purpose requires us to make such judgments, using the statis- 
tical analysis and our field experi^-ence , but we warn the reader that 
these findings are exploratory and tentative. 

The remainder of this section discusses the model's underlying 
assumptions in more detail; the latter part of this chapter presents 
our operational measurements of dependent variables at the classroom 
level. Our assumptions derived from several empirical lessons we 
learned in Phase I. First of all, as we suggested earlier, because a 
project is likely to^mutate during the implementation, its effective- 
ness after four or fiv6 years depends not only on its initial design 
but also, on its evolution. In turn, its evolution depends on char- 
acteristics of the innovation and its institutional setting. In other 
words', the implemented project should be treated as both an independent 
and a dependent variable in a system of relationships. Consequently, 
the effectiveness of implementation will be analyzed, on the one hand, 
as a function of the factors identified earlier, and on the other hand, 
as a factor affecting teacher change, continuation of method and ma- 
terials, and student improvement . 
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A second lesson concerns tlie project: *s cumulative effect on 
teachers after four o. five years of project operation. Teacljers may 
learn from and adapt to tlie project during imp Lernen tji t ion in ways 
the or igina 1 , designers may or may not have intended Such changes in 
teaching style or behavior constitute an outcome of the project as it . 
was inipiemented . Furthermore, improved student performance and con- 
tinuation of project metliods or materials depend on the teacher's style 
and behavior as they were both before and as a result of the project. 
Therefore, the change in the teacher's behavior can be classed as 
both cause and effect in the system of relationships determining what 
happens to an innovation. 

Although change in the style and behavior of teachers is a major 
potential effect of an innovation, it would amount to "t:hange for the 
sake of change'' if it had no effect on student performance. We con- 
sidered two serious methodological questions before we incorporated 
student performance into the model and analysis. First, the measure- 
ment issues are controversial — what should be measured and how to 
measure it — and we had to l .?rt measurements appropriate to this study; 
the next section discusser » .r operational choices. Second, one could 
question the appropriateness of measuring changes in student perfor- 
miMice bc( ause an innovation takes considerable tim'e to affect students 
positive, ; . . In fact. Phase I found that implementation difficulties 
during the first few- years prevent a valid assessment of how the in- 
novation ultimately might affect students. However, we bel-ieve student 
effects can be judged four or five years after an innovation formally 
began (the point at which Phase IT assessed projects). Despite method- 
ological reservations, then, this exploratory analysis considered improve- 
ments in student performance to be a separate project outcome (i.e., a 
dependent variable of the assumed system of relationships). 

The data analysis also treats continuation as an outcome — but 
only as one of several. That cautionary distinction is wise, for it 
would be myopic to regard continuation as the only outcome or even the 
miost important outcome of an innovation. Continuation is not always 
appropriate or desirable. For example, we observed in the field two 
opposite possibilities: One, an innovation may be discontinued even 
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tliuugh its othor ouLromes nro no table (perliaps because it liay served 
its purpose or tlie di.strict is unal)le to "afford" to sustain tlie In- 
novation); or two, an innovation may be continued despite poor re- 
sults (perliaps because the district is subject political or bureau- 
I'ratic pressure). For t hose reasons , continuation sliould be weighed 
along with, and sometimes against, other project effects. 

By using other outcome measures besides continuat\ion — viz,, im- 
proved student perf ormanet!? , teacher change, and implementation effec- 
tiveness — one can begin to get a sense of tradeoffs among desirable 
outcomes. F^r example, it may be, as the Ford Foundation discovered 
on projects they sponsored, that concentrating a project in one school 
maximizes gains in student performance but that spreading the project 
across many schools improves its prospects for continuation. 

Thus far we have outlined the components of our data analyKis 
model and why we chose them. We could have used other formulations, . 
however, that would be consistent with our theoretical conception of the 
process of change. We could have used a "reduced form" approach that . 
conservatively assumes that the "dependent" variables do not affect 
each other or that we lacked sufficient information to identify their 
interrelationships in any useful way. This reduced form approach means 
that the statistical analysis should • "estimate" the effect of the fac- 
tors on each project outcome separately, and then the estimates should 
2 

be compared.^ Because this conservative view has considerable merit. 
Chap.. 5*s statistical analyses were done in two ways: (1) using the 

2 

The reduced form approach can be stated algebraically as: 



^1 = 


f (X) 


+ 


^1 


^2 = 


f (X) 


+ 




^3 = 


f (X) 


+ 


^3 


\- 


f (X) 


-t- 


H 


^5> 


f (X) 


+ . 


^5 



where the algebraic i^ymbols have the same meaning as in the preceding 
footnote. 
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simple recursive* model of Fig, 4.1 and (2) using the reduced form that 
makes no causal assumptions about the relat lonsh Lps rimon^ the dependent 
variables. 

However, we could liave assumed a more complex causal system — 
e.^., that implementation caused clianges in teaching, and that changes' 
in teaching in turn affec.ted implementation. To test such "simultan- 
eous" relationships, we would need to collect data differently from the 
way in which we were able to. For example, we would either have.'to 
have collected data longitudinally or to have asked respondents to 
render such difficult retroactive assessments as how much they or their 
students changed during the first three years of the project as compared 
wi.th the first two years after the project. Moreover, a fully simul- 
taneous model would have required a sophisticated soc ial . science theory 
so tliat explanatoiy variables could be identified: Such theory does 
not yet exist. 

In summary, the, absence of a longitudinal data base .and of any 
firm theory meant that it would have been premature and probably spe- 
cious to assume more realistic but less tractable reja t ionsh ip5 than, 
those indicated "by Fig. .4.1. 

ME ASUREMENTS OF 'DKPENDENT VARIABLKS ; 

For tlie purpose of statiistical analysis, the dependent variables — 
the multiple effects of federally/ funded projects — had to be measured 
in terms that were comparable across a variety of innovations and in- 
stitutional settings. We could not use any single "objective" measure, 
of student performance a-? a test of a project's performance. For ex- 
ample, tffe use of a s tandard isied measure of achievement gains would have 
been inappropriate because projects in the Rand sample differ'ed in such 

dimensions as their educational objectives (e,g., cognitive aims versus 

. - . z 

affective aims for s tudents) , their curriculum areas (e.^.',. reading 

versus mathematics), their grade level (elementary versus secondary), 

and their techniques (e.g. , student-oriented ''individualization tech- 

niques' versus open classroom approaches). ' 

T6 obtain comparable measurements, we asked project teachers to 

assess the generalized effect of the I^Tinova'tion on themselves' and -bn 

* 
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their students. Their answers may be biased, perhaps tending to over- 
evaluate the teacher's and project's performance, in which case they 
cannot be considered accurate.- For example, the p'roject teachers re- 
ported 67 percent as the mean level of achievement of project goals. 
Outside observers might report a lower figure; but whether or not the 
figure is biased,' little harm was done because no use and jjo inferences 
were made from it per se . Our primary concern was not to judge project 
success but to explain the extent to whic* various factors affected- 
outcomes. Responses that were systematically "optimistic" but other- 
wise accurate did not invalidate our approach. 

To examine the nature of potential measurement errors in the de- 
pendent variables. Phase I included a validation study. Rand observers 
rated projects in 29 fieldwork sites (about 10 percent of the total 
sample) using the same instruments as those used, by teachers, principals, 
project directors, and superintendents. -The results, which are dis- 
cussed in Vol. II, show that Rand observers rated projects systemati- 
cally lower than did the participants. The "absolute" exaggeration or 
upward bias did not seen to be greater for some types of projects than 
others. This issue — whether the bias in the dependent variable is 
(a) only a systematic shift in . the answers for all respondents or (b) 
also . includes different amounts of exaggeration by different respond- 
ents—is an important statistical issue. If (a) were true, our explan- 
atory type of analysis uould be appropriate, within limits; if .(b) were 
true, the validity of the statistical analysis would be questionable. 
Judging by the results of the validation study, believe it is reason- 
able to assume, for exploratory purposes, thai r variation in t;hc 
operational measurements of our dependent variables was not severely 
contaminat,ed by a general upward bias in the self-assessments. 



Validity tests were not pursued in Phase II, except during pre- 
test 'Studies, : ocause' of research budget considerations. In the case 
of severe upw;tr:l bias, a "ceiling" effect, or artifactual skewiiess , 
can cause a variety of errors. For example, the variation at the. top 
end of the scale may be too collapsed" to analyze. In preliminary 
anslysis, we used a variety- of transformations and recodings of the 
data (e.g., using dummy variables) to test the extent of the ceiling 
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In summary, we used assessments by project participants to obtain 
comparable measurements of four project outcomes; implementation 
effectiveness, change in teaching, improvement in student performance, 
and continuation of the project's methods or materials. Tlie error 
introduced by these operational measurements cannot be precisely de- 
termint'd, but preliminary tests indicate that the errors are not 
major enough to invalidate our exploratory analysis. T!ie following 
section discusses the operational measurements of each of t!ie four 
types of project outcomes at the classroom level. 

Imp 1 emen tat ion Ef f ec Liveness 

The Title III projects in our sample addressed a number of differ- 
ent project goals. We .selected our Phase II sample to include only 
.those projects that, a i)riori, could be expected to influence class- 
room activities and student performance (e.g., we eliminated projects 
such as cultural enrichment and drug education projects). Yet even 
within these sampling parameters, the projects display sufficient 
variation — e.g., from remedial reading to open education to individual 
mastery learning to programs designed to enhance students' school- 
related attitudes — to make direct comparisons between specific project 
objectives mean ing^ ^^ss . As an index of effective project implementa- 
tion, we used a measure of the extent to which projects met their own 
goals, different as they might be for each project. Specifically, we 
asked teachers, principals and superintendents to estimate the percent- 
age ot project goals irhat were acliieved. This question is obviously 

problem; the formulat ions presented in tiie text were the* most appro- 
priate. In pretesting we" found that using a :icaie of seven (ordinal) 
categories rather than five categories greatlv reduced ceiling effects. 
Nonetheless,* the operational measurements remala ■ ^ibject to this 
problem. 
.4 

The specific question and statistics fcr the distribution of 
responses were : 

(Percent Goals Achieved) Overall, wheyi yL\ take hito account 
the (joals the pro.ject started with ayul the resouvces it hid, about 
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open to a number of response biases.^ For ex^nple, the Phase I validci 
tion study indicated thiit district respondents tended to be more opti- 
mistic- about the project *s "success" than were outside Rand observers. 
This systematic exaggeration was partly idiosyncratic (or random) and 
partly related to the respondent's position. For example, on the aver 
age superintendents rated projects liigher than did principals, who in 
turn rated projects higher than did teachers. 

l^iatever the reasons lor these differing responses might be, two 
conclusions for analysis follow. First, a measurement of project 
"success that combines the answers of respondents at different levi^ls 
of thi? school organization (teachers, principals, and superintendent.^": 
would fail to give a valid indication of the effectiveness of project 
implementation from any level. Second, both Phase I and .our fieldwork 
found that teachers' estimates correspond most closely with those of 
outside' observers . There are plauv^ible explanations for the apparent 
greater accuracy of teachers in judging the educational success of the 
project. For. one, they work in the classroom and can watch project 
impi-ementat ion and its re&iults with thti^r own eyes. Superintendents 
and principals necessarily have a more limited view, and furthermore, 

what percent of its goals would you say the project has ^achieved? 

< Teachers Principals Superintendents 

Mean ' 68.1 Mean =^ 75.9, Mean =79.5 

Median = 7/h . 2 Median = 80.0 Median =82.5 

Standard . Standard Standard 

deviation = 24.7 deviation = 18.3 ^^.jieviation = 15.6 . 

# responses 891 // responses = 155 // responses = 96 

Throughout this report, the exact wording of questions and the dis- 
tribution of res;:»onscc '^ill be presented in footnotes at the time the 
variables are first diccnpced. To identify the variables, their con- 
ceptual names are placed before the statement of their question. The 
responcien ts , of course, did not have knowledge of the variable names. 
For almost all variables in the analysis, the questions were designed 
to have a prima facie connection with the conceptual variable. 
5 

The Phase I survey analysis used five alternative measures of 
implementation effectiveness to test the validity of this question; 
all five were used in p:. ?liminary analysis. Given the. problems 
of comparability, simply asking about the percent of pr^^ject goals 
achieved appears to be the mosr operationally useful measure of im- 
plementation effectiveness. See Vol. II, p. 15, for an extended 
cisc'^ssion. 

• 7.1 
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are likely lo be swerved by bureaucratic: and political considerations 
in their as.3essnien ts , whether consciously or unconsciously. 

For these reasons, we used teachers' estimates as a measure of 
implementation effectiveness. it is important to understand, however, 
that this project outcome measure, which we jail percent goals achieved^ 
should not be seen as a proxy measure for project effort on students 
or teachers. For example, it is possible for a pro^t^t — particularly 
one with narrow objectives — to be highly "surcessf u 1 , *' perhaps achiev- 
ing 100 percent of its goals, but have little or no effect on student 
performance or teacher change. .The percent of project goals achieved, 
then, is only one outcome, and is meaningful only when combined with 
other measures. 

Change in the Pattern of Teaching 

Te^'cher change can have several meanings, with different impli- 
cations for classroom practice. Some projects, open 'c lassroom projects 
for example, focus primarily on changing t.ie way teachers organize 
t.^eir classrooms and relate to their students. Remedial reading proj- 
ects, on the other hand, do not primarily address the , way a teacher 
, organizes the classroom; instead, the aim is to give teachers new 



One reason for using percent goals achieved was to attain a com- 
parable measur-e across diverse innovations. It would t^e naive to 
assume that this measure eliminated all comparability issues, however. 
For e-xample, although projects differed in difficulty, a project achiev- 
ing 100 percent of its goals necessarily appears more "successful" than 
a project achieving 80 percent. To deal with this issue, we. followed, 
two procedures. First, as the chapter discusses, we did not equate 
percent goals achieved with "success,*' but rather used a variety of 
measures so that we could compare the effects of different factors 
(including project inputs) on each dependent variable; thus. Chap.. 5 
estimates how much ''percent goals achieved" affected teacher change, 
student improvement, and continuation. Second, data were collected 
from participants to indicate how difficult their project was; Table 
4.2 indicates the bivariate correlations of difficulty of project 
implementation, with other dependent variables. This variable was used 
as a control for all preliminary runs of analyses presented in Chap. 5, 
but its inclusion did not change the substantive results. In short, we 
have tried- to- mitigate the noncomparabi 1 i t ies in our operational meas- 
urement of percent project goals achieved. 
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curricular skills they can use with students having particular learn- 
ing problems. Furthermore, although a project might have been deliber- 
ately planned to introduce specific types of other changes, teacher 
change might have been an unanticipated consequence. A single measure 
of "teacher change," then, would be likely not only to mask particular 
project effects on teacher behavior but would also be of only limited 
utility \A understanding what kinds of teacher change are associated 
with what kinds of change in student performance. 

To differentiate among the dimejisions of teacher change, we asked 
both teachers and principals to assess the extent to which classroom 
behavior had changed along a number of dimensions. Table 4.1 lists 
the questions we asked and the distribution of responses. 

Our assumption was that these questions spanned different areas 
covered by different types of projects. To develop a single measure 

comparable across all types of innovations, we added together the re- 

^ " 7 

sponses of each teacher on ;^f our of the questions; we call this opera- 
tional measure tw^ total teacher change caused by the project. Table 
A. .2, which presents the in tercorrelat ions among the dependent variables 
used in Chap. 5, shows that the responses to the teacher change vari- 

o 

ables are highly correlated (ranging from .63 to .40). This high 



The second question in Table 4.1 is more relevant to secondary 
school teachers; no primary grade teacher^ answered it. When we took 
into account the pattern of no responses to this question and to the 
first question about basic skills, the' two questions were highly cor- 
related. Accordingly, we dropped the second question from further 
analysis. 
8 

Except for percent goals achieved, ""he variables in Table 4.2 
are defined from questions that use seven- loint scales. These scales 
are ordinal, but we use them as if they we: e equal-interval (cardinal) 
measures because more powerful statistical techniques^ (viz . , multiple 
regression) can be applied. -For a justifies ..on for this approach, 
see Robert P. Abelson and John W. Tukey, "Efficient Utilization of Non- 
Numerical Information in Quantitative Analysis: General Theory, the 
Case of Simple Rank Order," Annals of Mathematical Statistics, Vol. 34, 
1963, pp. 1347-1369. 

The use of scale variables in any empirical work can be challeng'ed 
because the meaning of a scale response to a question may not be com- 
parable ^across individuals (i.e., "interpersonal comparisons" may not 
be meaningful). For example, people may give, different meanings to 
the end points or to the "distances" between different parts of the 
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Table A . 1 
QUESTIONS ONi TEACHER CHANGE 



Teachers were asked ch'i Cjllowing question: each of the teaching activities 

listed bel< '^), please i.-idicate Jiow much ijou think you have charujed the way you do 
things. Wc are interested in all changes, whether or not they were directly' 
intended by the pro j set," 



Very 
Large 
Change 



Number 
No of 
Change • Responses 



Teaching of basic skiils 
(reading* writing and/ 
or math) 

Teaching of one or more 
other specific curricu- 
lum areas (e.g., sociai 
studies, science, art) 

Recognizing and attend- 
ing to individual students' 
special learning problems 

The way you organize your 
classroom 

Working with students in 
the area of affective 
development 



12.1% 16.5% 23.5% 16.5% 7.A% 10.6% 13. A% 



9.3 

20.3 
17.2 

13.3 



16.0 

23.8 
22.6 

22.7 



18.0 

22.6 
20.6 

2A.8 



15.8 

12.0 
13.8 



8.9 

5.9 

7, A 



17. A . .7.1 



12.3 

6.6 

7,3 

6.3 



19.8 

8.8 
11,1 

8, A 



1057 

9A5 

1021 
1012 

lOOA ' 



Principals Rave the following frequency of response to the question: 'Uh'or each of 
the teaching actii)iti*^s below), please uidicatc how rjuch chatigc you have obiuiroed 
in the way projee t teciche rs i n j e > i e /v 1 1 act ually do t h i nrg r, in their cl ar: n tons 



Teaching of basic skills 
(reading, writing and/or 

Teaching of one or more 
other spec if ic . curricu- 
lum areas (e.g., social 
studies, science, art) 

Recognizing and attending 
to individual students' 
special learning problems 

The way teachers organize 
their classrooms 

The way teachers work with 
students in the area of 
affective development 



10.9% 

8. A 

27,1 
19,2 

17.3 



20.5% 32.7% 16.7% 7,7% 



6.A% 



2A,7 

■38,7 
37,2 

35.3 



30.5 

20,7 
16,7 



30.1 
__J 



1A.3 

6.5 

12.8 



9.6 
I 



1.9 
6. A 



3.2 
I 



1.3 
1.9 

2.6 



5.1% 

6,5 

3.9 
5,8 

1.9 



156 

16A 

163 
165 

167 



3. 



NOTE: The five variables in the teachers' questionnaire have the following names 
throughout the text: Change in Teaching Basic Skills', Change in Teaching Specific 
Curriculum, Change in Dealing with Individual Learning Problems, Change in Organizing 
Classroom, and Change in Dealing with Affective Development. 
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Table 4.2 



flIVARIATE CORRELATIONS' OF DEPENDENT VARIABLES 



Variable ■ ' . ' Variable Number 

Number 

(1) Percent of project goals achieved 

Change in teaching basic skills 

(3) ■. Change in .dealing with individual 

learning problems 

J^)-' Change in organizing classroom 

(5) Change in, dealing with aff. dev. 

(6) Change in student achievement ' 

(7) Improvement in student behavior ' 

8) Continuation of project methods ^1 A7 // n 

. Continuation of project materials , ! ' ' ' ' • " 

^''^ ^^^^-^^y of P-J- impWati^^^ 

NOTE: Number of observations is 812. • 



(1) 


(2) 


(3) 


(^) 


(5) 


(6) 


1.00 












.32 












.29 


.57 










.32 


.59 


.61 








.27 


.^0 


.63 


.59 






.52 


.36 


.33 


.26 


.20 






.25 


.,32, 


.30 


.34 


.61 


M 


.45 


.40 


.51 


.47 


.44 


.50 


.30 


.77 


.31 


.26 


.33' 


■ 25 ; 


.10 


.03 


',05 


.06 


-.2] 



nt, ho^^ difficult has this project been to carnj out? 
f 2n 17^ 12^ , 11'^ 6^ 



7 

Very 
Difficult 
Number of responses is 1033. 



6 5.4 3 21 

Not at all' 
Difficult 
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degree of association probably comes from a combination of a "halo" 
effect and a valid correlation among .the types of changes produced 
by an innovation. The correlation among these variables is, on the 
one hand, high enough to Indicate that the total teacher change meas- 
ure makes sense; on the other, it is not so high as to preclude the 
possibility of discerning different effects from the factors for the 
different dimensions of teacher change. The next chapter analyzes both 
total teacher change and each of its components. 

Improvement in Student Performance . . 

Change agent projects typically introduce- activities and materials 
new to the district and therefore require time to "settle in" b(5fore 
lt)nger-run effects on student performance can be assessed with any 
accuracy. For example, teachers acquire and practice new skills or 
behavior i^ver the course of project implementation. For that reason, 
we deferred questions about changes in student performance to the 
Phase II survey. 

Change agent pro jec L's can aim tov\/ard cognitive or, affective im- 

9 

provements in student performance, or both. More than three quarters 
of the teachers in our samp 1 e ind icated that their projects aimed at 
both. To measure change in these two dimensions in comparable ways, 
wc asked teachers to assess the relative extent of change in both 

scale.. Our approach could he justified by assuming that pur respond- 
CMits* utility functions are approximately cardinal and fairly similar 
(i.e^., Che distribution of utility functions has a clear central 
tendency and a relatively small variance)." This assumption may be. 
plausible for many of our variables because the population of teachers 
is relatively homogeneous and used to dealing with scale variables. 
More 'research is needed to test this assumption. 
9 

' We asked teachers to indie ace the student object ives for . their 
change agent project. Specifically, we asked (Improvement in Student 
Achievement): "/t6' yon understood the projnoty was it primarily aimed 
at your students' Qognitivc development y affective development or both?" 
The distribution of the 980 teacher responses was: 

Cognitive development only 11. 2% 

Affective development only 10.2% 

Both 78.7% 
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dimens ions . "^^ The distribution of responses for both questions is 
quite varied, but Table 4.2 shows that these measures are highly 
correlated (.61). 'As we did with teacher change, we added together 
the two student change measures to give a single measure, which we 
call tiie relative extent of total iniprovemeyit in r.tudent per^forrnanae , 
Chapter 5 analy7.es the combined measure as well as the separate meas- 
ures of achievement and behavior. 

Although Lhe distribution of answers to these questions suggests 
that teachers were discriminating in their responses, teacher percep- 
tion of change in student ^performance is obviously open to a number of 
biases. Teachers who had invested much time and energy in implementing 
a project might be reluctant to admit that their efforts were fruit- 
less, which may explain why approximately half the teachers reported 
that their projects had markedly increased student academic achieve- 
ment. They may have been right, of course/, considering the select 
nature of the projects in our sample, but such results are strikingly 
good compared with those reported in the research , literature and in 
evaluations of special projects. (For example, many studies in search 



"^^The, specific questions and distribution of the 982 teacher re- 
sponses were:^. 

(Improvement in Student Achievement). "What impact has 'its 
project Jiad on studeyit academic achievement?'^ 

5.6% 12.9% 3J.1% 19.6% 23.7% 3.1% 1.7% 1.6% 7% 



9 8 7 6 5 A 3 2 1 

Much higher o No change Much lower 

achievement achievement 

(Improvemen t in Studtmt Behavior ) What effect ha.s this 
project had on .student attitudes or behavior?'' 

9.9% 17.7% 27.3% 20. A% 17.3% 2.9% 2.8% 1.2% 0.6% 

L L 1 I L_ I I • I 

9 8 7 -6 5 „ A 3 2 I 

Very positive No effect Very negative 

effect • effect 

^■^Volume T provides an extensive review of the literature. 
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of "successful" projects riraong a sample of candidates have been able 
Lo identify a number that only barely exceeds what would be predicted 
by c hance . ) None the less, teacher responses var led widely, wh ich in- 
dicates that it would be fruitful to analyze the oouPces of variation 
and tlie relationship to other project outcomes. 

This' measure is also open to measurement error because it reflects 
teachers' perceptions of student change,' not objective, standardized 
test results. For example, there is a. sizeable liternture that sug- 
gests that teachers systematically give higher marks to middle-class 
or "social -conforming" students. Even so, we believe that teachers' 
perceptions are useful for at least two reasons. First, since project 
objective?; differed widely, between-project comparisons of student 
gains on standardized instruments are not meaningful.' A comparison 
between one stanine's growth in math acli ievemeht and a t;en percentile 
gain in reading proficiency, for example, tel4.s us little. Teacher 
assessments of project-related student gains, however, permit us to ^ 
assess relative' if not specific differences in project effects on 
students. Second, it can be argued that teachers' estimates of student 
gains measure more enduring or comprehensive project effects than can 
standardized achievement or atti tude measures . Because of their 
close involvement with their students, teachers are likely to make an 
integrated assessment that represents solid student gains, not simply 
enhanced test-taking facility. Further, it is to be expected that 
teachers draw on standardized student outcome measures in making their 
assessments. Teachers* perceptions probably derive from a synthesis 
of information about change in student performance from a number of 
sources; consequently, it can be argued that their perceptions more 
accurately represent solid' improvement in student performance — in 
reading skills, for example, as opposed to narrow gains in vocabulary, 
which may prove ephemeral. , ^ 

In summary, although teacher measures of student gain are subject 
to a number of response biases and comparability problems, it seems 

For a review of the literature, see H. A. Averch et al.. How 

. EffcQtivc Is Sohoolinrj?' A Critical RevieiJ and Synthesis of Research 
Fiyidiri/jSy The Rand Corporation, R-956-PCSF/RC , March 1972. 
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reasonable to assume for the purposes of exploratory analysis that 
those difficulties dy not invalidate our analysis. 

f 

£li^LJ^ilL Id^LL^JL^^iL^J^^^^^^ t:h e Classroom Level 

11 one asks n school official, "Has tfiis project been continued?", 
the answer would likely be, "Yes." But if one probes or visits class- 
rooms, a mow d i f ferent i^ited view emerges; the extent to which project 
methods or materials continue to^be employed can vary enormously. 
Sometimes continuation is no more than pro forma; perhaps the innova- 
tion's methods are wholly or mostly discarded, but. the materials con- 
tinue* to be used, at least to some extent. Other projects are fully 
continued; the staff have assimilated the projectVs methods and employ 
them extensively in their teaching. Therefore, in choosing the Phase 
II survey sample of purportedly continued projects, we assumed that the 
true extent of continuation would vary. 

This assumption was accurate. The survey asked teachers to esti- 
mate the relative extent to which they were ourrently applying the 
basic ideas and^methods of the project in their class-tooms. ("Cur- 
rently," in this context, meant two years after the end of federal 
funding for most teachers in the sample. )^'^ The distribution of re- 
sponses varied, as expected: 17 percent indicated they were continuing 
the project to a "very large extent" (the highest point in our seven- 
point scale), and 5 percent indicated they were not continuing the 
project's methods at all. Teachers were also asked about their current 



13^, 

The specific questions and d-^ stribution of the 980 teacher re- 
sponses were: 

(Continuation of Project Methods) ''To what extent are you 
presently applying the basic ideas and methods of the projeot 
in your classroom?'' * . ' 

17.2% 27.0% 23.0% 16.3% 6.8% 5.r% 4.5% 



7 6 5 4 3 -^ 2 1 

To a very To some Not at 

large extent extent all 

Notice that' this question deals not with official continuation of tiie 
project qua project but with continuation of the project *s content. 

8U 
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use of the project's materials Uause materials can be continued with or 
without the project's methods. Again, the responses varied widely: 
19 percent indicated they still used all of t;he project^s materials, 
and 15 percent indicated they were using none. These measures con- 
stitute^our operational definitions of continuaTi^n at the classroom 
level. The statistical analysis in Chap. 5 explores why some teachers 
continued their projects extensively and others did not. 

We interpret these measures, particularly the question about 
•methods, to represent the degree of the teacher's assimilation of the 
project's precepts. One test of the validity of this interpretation 
is to compare the teachers' responses to the question about their 
cvAVi'^nt actual- use of the project with their inclination to use the ' 
innovation elsewhere were they to take a different teaching job.^^ If 



14 

The question and distribution of the 1002 teacher responses 

were: 

(Continuation of Project Materials) "What propovtion of project 
materials are you now using in your classroom?" 

All of them 18.6% 

Some of them 32.6% 

A few of them 23.2% 

None of them 14.6% 

Table 4.2 shows that the extent of continuation of the project's 
methods was correl.Jted .57 with the cdnt inuat-ion of project materials 
We expected these measures to hf highly correlated; however, they are 
not so highly correlated as to Irevtent a separate analysis of each 
measure . ' 
16 ■ •. 

The specific question ahd distribution of the 1016 responses 

were: 

(Inclination to Continue Project) "Tf you lotwc to take a 
teachtng job in another school or district that had not tried 
this project; how inclined would you- be to try to implement 
its methods in your classroom?" . . 'i 

28.2% 22.1% 18.2% 15.0% 4.2% 4 4% 7 9% 

L L___J L__.J i p 

6 5 4 3- 2 il 

Very Moderately Not] at all 

inclined inclined iriclined 
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teachers cont inued to use the proj ect ' s Snethods solely to conform to' *^ 

school policy, we would not expect their current use to be positively 

related to their inclination. But the correlation is an impressive 

.61. This degree of agreement supports the interpre tat.ion - that bur 

continuation measure represents a surrogate for the teacher's assimila- 

18 

tion of project precepts. 

As another test of validi ty v-^we asked principals to give us their 
opinion about whether the project at their school would eventually die 
out or would bect^me integrated into the teachers' classroom prac t ices . "'"^ 



■ 17^ . . . 

It IS inCeresting that con t inuat ion . of ma t er ials is correlated 
.A3 with the teachers' inclination to continue them elsewhere. We 
wonder if this lower correlation (compared with the .61 correlation 
for methods) reflecCs the possibility that materials are more likely 
than methods to be continued by teachers because of official school 
policy. '. 
18 ■ 

The survey also asked teachers to indicate the extent to v/hich 
their involvement in the project had made their job more (or less) 
satisfying. The variable was correlated .57 with continuation of 
project methods and .37 with continuation of project materials. These 
results offer further support for the validity arguments made above. 
The specific question and distribution of the 992 responses were: 

(Job More Satisfying) ''Would you say your involvement in the 
project has made your job more satisfying?'' 

17% 18% 22% 15% 22% .2% 1% ^1% 1% 



■9 8 7 6 5 A - ^ 3 ' 2 1 

Much more * No more or Much less 

satisfying less satisfying satisfying ' 

19 

^(School Continuation) '^Qver the nex't few years ^ whioh (of 
the following) possibilities do you think is most likely to happen 
with respect to project methods among teachers at this school?" ^ 

The methods will be dropped 2.0% 

Some or all of the. teachers will use project methods; 

but methods will not be used extejisively 48.4% 

Some or all of tlic teachers will use project methods 

and they will generally be Used quite extensively 49.7% 

(Number of responses = 98.) The 50-percent "continuation rate" should 
not be considered as representative of all Title III projects; our 
sample is not a random sample but highly selected toward purportedly 
continued projects. Thus, one would guess that the continuation late 
of all Title III projects would be considerably below 50 percent. We 

82 
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The split between the two , "futures" was about even. By. averaging 
the answers of teachers, at a school to the question of continuation 
of project methods,, we arrived at a rough estimate of the correspon- 
dence between, on the one hand, the principal's sense of the stability 
or endurance of the innovation over time and, on r.Iie , other hand , the 
teachers' statements about how extensively they were using the project' 
methods. The average score of the teachers' answers seems to be 
. significantly higher for the group of enduring pr.>ieccs frhan for proj- 
ects the principal believed would eventually die out.*" Though this 
evidence is far from a conclusive test o.' /^alidity, it lends credence 
to the assumption that the teacher's srtement about the extent of 
their current use of project methods represents a measure of the proj- 
ect's longer-term use — that^is, a measure of the project's assimilation 
into the staff^s standard pedagogical repertoire. 

In conclusion, the opera t^^iona 1 measurements of continuation at the 
classroom level — the extent of tenchers' use of th(- project's method.*; 
and materials some two years after the end of federal funding — seem 
valid for exploratory purposes. 

SC OPE AND LIMITA T IONS OF THE FINDINGS 

f This study addresses complex and difficult issues. It aims to 
identify a set of factors that determine the continuaticVn of innov.i- 
tions after federal funding ends, and to do so for a wide range of 
inn(jvations in diverse local settings. The absence of a well-defined 
theory of the process of educat iona I , change moans, that our findings 
cannot be treated as definitive. Moreover, our sample, which was 



suspect that, even for our sample, 50 percent might be optimistic. 
(Note, however, that an upward bias on the part of teachers should not 
seriously^ affect the analysis using the arbitrary seven-point scale 
of relative extent oT continuation.) 

^^Using a scven-point scale (with., "Co a very laVge extent" as- 
signed a seven), the average score of the (average) teacher answe.r:'. 
within a school was 5.9 for projects in which the principal thought 
the- teachers would continue to use che project's methods extensively; 
tlU' score for the other group was 4.2 The "difference" of these means 
appears to be significantly higher. However, ^he statistical test is 
.n{)t straightforward because of the different iuimber of teachers repre- 
S(?nted for each project. .* " . 



-hi- . 

solected to fcu-i.*" ] oont iinint ion , is no t • rep rosen la 1 1 ve of all Title 
ill pr(\jci!ts, lot alone of all innovative projects. Consc juen 1 1 y , 
tlie K-'"^^"'^ i i ^nb i 1 i I y of our results can be quuscioned . To achieve 
comparability aoros.- Mors c,ad jirojects, we measured project ef- 

fects and continuat i "con ten t -free " scales. Such measurements 

Hie open to valid ( .in. Finally, tiie data analysis model makes 

simplifying assumptions in order to excimine systematically the com- 
plex relationships among factors affect ing change agent projects. 
The-e caveats should be kept in mind as the reader considers the re- 
sults of Chaps. 5 and 6. 

Despite these caveats, we believe the f indings suf f icient ly valid 
to be used bv policymakers and researchers. 
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Chapter 5 
FIN DINGS AT TUE CLASSROOM LKVKL 

\^\y ere some innovations effectively imp 1 emen ted and continued 
while others fall short of their original promise? Uliat character- 
istics of innovations and of their host school districts explain the 
disparate experiences and outcomes at the classroom level? The pre- 
ceding chanters provided the groundwork for examining these questions. 
Here we present statistical analyses, along with interpretations 
based on tieldwork experience, that explore answers to these ques- 
tions at the classroom level . 

This is a long chapter. Consequently, fol. lowing a brief sta- 
tistical iface, we present an overview of the findings to equip 
the reader with a guide to the subsequent discussion. 

STATISTICAL PREFACE 

Tables 5.1, 5.2, and 5.3 present the results of* a statistical 
analysis designed to explore the extent to which the factors iden- 
ti'^ied in Chap. 2 account for the variation in projecrt effects and 
continuation. Using the simple causal model discussed previously, 
we ran a series of multiple regressions that relate operational 
measurements of the various factors to our measures of project 
effec:ts and continuation. 

The first column of Table 3.1 displays the average effect of 
each factor on implementation effectiveness — i.e., the teacher*^' 
assessment of the percentage of goals the project achieved — control- 
ling for the presence of the other factors. The statistic repre- 
senting these effects is a standardized regression coe f ficient , 
which denotes an estimate of the relative weight (ranging from -1 
to -fl) that each factor has on the dependent variable for our data.^ 



The s t andardized regression coe f ficient is used in these 
tables and elsewhere in this report as an aid to interpreting the 
regressions. It can be deceptive, however, particularly when dummv 
variables are used (Table 5.1 has three dummy variables^ Overall 
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Table 5. 1 

FACTORS AFFECTING PROJECT EFFECTI VENKSS AND CONTINUATION: RFDUCED-FORM 
REGRESSIONS FOR MAJOR DEPENDENT VARIABLES 
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l-ACTO S AFKECTINC; PKOJKCT KFKf'CTfVKN'KSS AND CONTTNLIATION : REDUCI^n-R)KM 
RI'GRFSSroNS KOK SKCONDARY nKPFNDFWT VARIABLFS 
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KA(.T(1KS AFFKCrr.MC PUOJKCT FKKCTI VKNKSS AND 
CONTTNIIAT KIN: KKCUKSTVK RKCRF.SS KINS 
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For example, the first entry in the first column is .03, which indi- 
cates the relative increase in implementation effectiveness that 
would result from a one standard deviation increase in project fund- 
ing level. This amount of change is relatively small. Indeed, con- 
sidering the random error this sample of projects is subject to, • 
this effect is not significantly different from zero (the asterisks 

in Table 5.1 indicate that a variable is significant for a standard 
2 

2-tailed t-test)." In other words, controlling for the other factors 
and assuming the basic theoretical structure is correct, the data 
suggest that the following hypothesis should be seriously enter- 
tained: the funding level of a project has a weak and not signifi- 
cant influence on implementation effectiveness. 

2 

The percentage of variation, R , explained in the dependent 
variable "Percent Project Goals Achieved" by all the factors listed • 
in Table 5.1 is .50, which is a high level for this type of survey 
data. The number of observations (teacher responses) used in the 
regression of tol. 1 is 499, the same figure used for all the 



Change Required in Teaching, Staff Paid for Training, and Elementary 
School). Appendix A presents the actual regression coQfficients and 
standard errors for these data; it displays the means, standard devi- 
ations, zero-order correlations, and partial correlations of vari- 
ables used in Tables 5.1, 5.2, and 5.3. Appendi.x B presents an anal- 
ysis of r -ariance o" teacher responses for' each of the variables 
used in the tables; the analysis partitions the teacher variance into 
the part explained bv the teacher's school or bv the teacher's project 

To simplify the presentation of statistical material, the tables 
indicate only whether a coefficient is significant at the .01, .05, 
or .10 level (for a standard 2-tailed t-test) , instead of displaying 
more detailed information, such as the coefficient's probability- 
value. Judgments about statistical significance should not be made 
mechanically. Nonetheless, for our data, a significance level of 
.01 represents a relatively "sur? bet" that the coefficie'nt is dif- 
ferent from zero; a significance level of .10 is on the borderline 
and, in these instances, wc use both statistical and nonstatistical 
information to arrive at a balanced judgment. Testing whether a 
coefficient is significantly different from r,nro is not always the 
relevant issue. For example, we will occasionally be interested in 
testing whether one coefficient is different from another, ur whether 
the addition of one or more variables to a regression significantly 
affects the results. Though, the text will not display the diverse 
statistical testing done in these cases, footnotes will supply rele- 
vant infomat ion. 
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regrossions in Tables 5.1, 5.2, and 5,;]-. This number is far less 
tlian the total number of teacher responses (1072) because we deleted 
any respondent who did not answer any one of the questions associated 
with the dependent or independent variables. This use of a reduced 
sample did not seem to bias- the fLndin^:.s significantly. 

Kach column In these tables presents the findings o f regrnssions 
for each of the dependent variables discussed in Chap. U. T^iough the 
independent variables (or factors) listed in the tables were intro- 
du(^ed in Chap. 2 in conceptual terms, most of their operational 
measurements were not. We will describe the specific measurements 
wlu^n the findings for each i ndependent variab le are discussed and 
interpreted. 

Table 5.2 differs from Table 5.1 in that the latter presents 
results for thf major dependent variables — implementation effective- 
ness (percent project goals achieved), total teacher change, total 
student improvement, continual) of project methods and continu- 
ation of project materials — whereas Table 5.2 presents results for 
the components of total teacher change (in teaching basic skil Is , 
dealing with individual learning problems, organiz.ing the class- 
room, and dealing with affect ivc development) and the components of 
total student improvement (in achievement and behavior). Both of 
these tables have the larel "reduced form regressions" in their 
titles to distinguish them from Table 5.3, whose regressions follow 
the "recursive" model described in Chap, 4, A reduced form simply 
means that Liie J.epehdent variables are assumed to depend only on the 

3 

It IS generally regarded as good statistical, practice to delete 
a respondent f : om the analysis if the respondent had a missing value 
on an independent variable used in a specific regression; the alter- 
native procedure of "simulating" the missing answers can create r 
bias than it eliminates. Howaver, since' each regression in the alM^/o 
tables was run independently of the others, there was no statist! il 
reason to Le je/vations from one regression because of a mi 

ing value m a dependent variable of anot!-?,er regression . Nonetheless, 
we followed Lais procedure — ciH therefore ha\.r the same number of 
observations for each regr.:>sr; i- -to save computer costs. Extensive 
use of preliminary versions r.f the models used by the tables indi- 
cated that the pattern of significance v.iries very little for the 
larger samples. 
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independouL vari.al)les; i.p a recursive model tho dependcMU variables 
depend on both independent and dependent variables (according to i lu- 
st rue tore outlined in Chap. 4). 

More speci f ical 1.V, the first column (stub) of Table S . 3 is thc^ 
same as tliat in Table 5.1. The second column of each table includes 
the same independent variables, bu^ Table 5.3 also has the percent 
project goals ach ieved (implementation ef feet iveness) as an "inde- 
pendent" variable. The third, fourth, and fifth columns of boLli 
tables have the same independent variables, but Table 5.3 lists 
percent project goals achieved and total teacher change as additional 
"independent" variables. The subsequent discussion of the findings 
revealed by these regressions compares the results using the reduced 
form with those using a recursive system in order to explore these 
complex effect's more fully. 

For the same reason — to develop a richer understanding of a 
complex reality — a variety of suhanalyses and statistical tests were 
performed involving variables related to tli*. independent variables 
used in the regressions of Tables 5.1, 5.2, and 5.3. These addi- 
tional analyses will be referred to when we interpret the substantive 
meaning of the regression for each independent variable. 

Tor the sake of brevity, we will not hereafter repeat or dwell 
on the formal meaning of each statistic. Nor will we reiterate the 
appropriate caveats about the nature of our data and methods. Nouct-- 
t he less, the reader slo'uld keep jn mind that tin's is an exploratory 
analysis. 

OVERVrKW OF FINDINGS AT CLASSF^.QQM LEVEL 

The detailed findings and interpretations presented below follow 
the structure of Tables 5.1, 5.2, and 5.3. The effects of the three 
groups of factors — federal input , project characteristics , and 
institutional setting — on the various measures of project imple- 
mentation and continuation wll! be discussed in turn. This presen- 
tation discusses each set of variables at some length; therefore, 
before proceeding with detailed interpjretat i ons , we offer the follow- 
ing (verview of the findings. 
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Federal Input 

I)i ffcriMirus I'll tlu' 'nnuninl of Title II ! funding had 1 Ltlie con- 
sistent or s i i i ^'«JiH ui'fect on project outcomes or on continuation. 
Although federal money (Mearly mattered to local school, districts in 
a number ol ways — particularly, their ability to undertake a projGc:t 
at all — any change wrought in district practices depended on what 
the district did with tiie funds, not on dollar amou 

V ro j ec t Cha rac i vr i s L i cs 

ji^LfL^^L^.^''.^l:'^j....^It'_yif^i^, • ■ The . ducat Lonal method or technique of an 
M^'i )vat ion had littie effect on project implementation, ntitcome, and 

:it inuat i on . Because of differences in project implementation 
strategies and in local ins t i t: t lona 1 factors, similar methods pro- 
duced dissimilar results across sites. dene rally ^-peaking, there- 
fore, no one method was more likely than another to fulfill its 
promise, particularly in the long term. In terms of the "success" 
of a federal investment, then, it mattered less what the project 
was than how it was carried out. 

%l^.ll?y. } Propos ed Chan ge . Projects attempting a broad scope of 
change were nof more or less likely to break down, and were somewhat 
more likely to he continued, than were less ambitious or more nar~ 
rov'lv defined projects. lncUa"'d, projects that required significant 
cluuige in overall teaching style and that called foi r.y.'-.rn effort on 
the part of staff induced significantly more teacher change. Ambi- 
tious projects therefore were not a "liability" from a funding point 
of view. Clarity of project goal.^ was important in the implementa- 
tion of all projects, however, particularly t hose a t t emp t i ng a broad 
scope of chang.e. For such pr(^jects, staff uncertaintv about what 
they were expected to do generated severe iniplementat ion problems 
and contributed to project demise once federal funding ended. Clar- 
ity of goals could not be "given" to a project staff at the out.f::, 
however; it had to be achieved in the course of imp! ementjat ion . 

^^P. lemeri t a t i( mi_ S t r a t e g i l\ s . The i m p 1 em e n i a t i n strategies cli ( t s e n 
for a project strongly affected its outcome and itr; 'continuation. In 
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particular, well-conducted training in how to use project methods 
and materials enhanced implementation and improved student perfor- 
mance., (The quality of training, not the quantity, made a major 
. di f fereniie . ) Plveri ^'a' ^fi training appeared to have only short-term 
effects, however, unless it was given in combi nat ion wl th staff 
support activities. Spec i f i ca.l ly , when activities such as assistance 
in the classroom, the use of outside consultants, classroom observa- 
tion, and staff meetings were done effectively (especially by giving 
prnctical support), they provided the feedback and timely assistance 
the staff needed in adapting project strategies to their own class^ 
rooms. Such assistance also helped create the climate and moral 
support necessary to motivate teachers to expend the e'fort that 
made projects work. The crucial ity of wel ] -executed staff support 
activities was evident in the strong positive effects they had, as 
a group , on a 11 the dependent variables : percentage of pro j ect 
goals achieved, teacher change, improved student per fnrmance , and 
continuation of project methods and materials. 

Teacher participation in project decisions enhanced implementa- 
tion and heightened the chances for continuation. Tt did so because 
it promoted a "sense of ownership," especially when thp staff devel- 
oped part or all of their ovn materials. 

Pay [rr training did not contribute to continuation and, in 
fact, v;as negatively related to the Dercentage of project goals 
acliieved and total student change. 

The local allocation decisions that are part of project imple- 
mentati(^n — that is, the number of project schools and the funding 

por student^ — generally had no significant relationship to 
project outcome or continuation. (There were two exceptions; More 
concentrated funding per student was positively associated with 
improved student performance; and the greater the number of project 
schools, the higher was the proportion of project materials con- 
tinued.) 

*. 

Institutional Set ting 

Organizational Climate and Leadership . Leadership was a vital 
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factnr at both the school and the project level... Effective imple- 
mentation required a good project director and a supportive jjchool 
pr'ncipal. Rut cont ImMt i on depended less on having had an effec- 
t.iv/e proiecr dLroctor than on the early and lasting support of 
principals. The organizational climate of the project — the quality 
of working relationships—strongly affected the percentage of goals 
achieved and projec-t continuation. 

School C harac tuns tics. The typr. of school had little or no 
relat ionship to project outcome or continuation. The exception to 
this general finding was the difficulty encountered by secondary 
school projects both in achieving effective impleme.ntation and in 
promoting teacher change. 

Teacher C ha r a c t e r i s ^t u j^;^ . Teacher characteristics had major 
effects on project outcomes. Above all, teachers' sense of efficacy 
emerged as a powerful explanatory variable; it bad major positive 
effects on the percentage of project goals achieved, improved 
student performance, teacher change, and continuation of project 
methods and materials. The teacher's years of experience, in 
contrast, had a consistent negative relationship to project out- 
comes; experience . was negatively related to the percentage of pro- 
ject goals achieved, teacher change, and student improvement. The 
teacher's verbal ability was positively associated with improved 
student performance, but otherwise did not affect implementation, 
teacher change, or continuation. 

In the pages that follow, .elaborate on each of these find- 
ings in turn. A more detailed su:nmary appears at the end of the 
chapter. 

FEDERAL INPUT TO C H ANGE AGENT PROJECTS 

As Chap. 2 suggested, money is the primary fee ml input iu 
Title III projects. The quantitative analysis of this section 
explores only one aspect of the role of federal mor-o v- -n 'Mrcly does- 
the amount, "seed monr_ matter for project'^ef feet and continua- 
tion at the classroom level? Because this chapter focuses on 
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(^Inssrnoni i» f foe I s , must deafer iinLil Chnp. 6 an analysis of sonu» 
lit tlu' issues concern inv, liow .//.t/ /v', use foderal money. Nonetlie- 
less, Llu» information provided here aboiit Llje re 1 a t ionslii p between 
thi' level of projfct funding and changes in pro j ect 'outcome provides 
clues to larger cpiestions about federal policy.. 

The evidence suggests that, within the broad range' of funding 
levels in'our sample (!^10,0n() to $350,000 for the year of highest 
funding), differences in funding had little effect on project 
results. In particular, the first rows of Tables 5.1 and 5.3 
show ItUv' and insignificant regression coefficients across all our 
measures of outcomes: the percent of goals achieved by the project, 
tlu:^ amoiint of teacher change resulting from project participation, 
improvem(>nt in student performance, and con t inua t ion • o f project 
methods or materials. More expensive projects., in other words, 
were no more likely than less expensive ones to be implemented 
effectively, to lead to change, or to be continuGfl at the classroom 
lt»vel. if auy significant effect of the level of funding can be 
discerned at all, it would he a nega t i ve t endency for continuation' 
of project materials (the coefficient of funding for continuation 
of project inaterials is -.08 and is almost significant' at the .10 
level in Table 5.1). That is, more heavily funded projects had a 
.slight tendency to have their materials discontinued by project 
participants after the end of federal r-mding. We observed this . 
case in the field in situations, for e.%ample, where school districts 
had ad(.ipted, on a large scale, curriculum packages such as SWRL 



The variable used for funding 1(Wl'1 in these regressions is 
the largest yearly amount of funds for the project from all grants, 
as Chap. 3 described. Its distribution is highly skewed for our 
sample, with most projects funded at about $100,000 to $120,000 in 
their year of highest funding. To compensate for this skewness, 
the logarithm of the variable is used in all regressions. 

■^Tahle 5.'2 shows that the funding level is significantly posi- 
tively related to change in teaching of basic skills. This effect 
comes from heavily financed remedial projects located primarily in 
. Title I elementary schools. Their pedagogic approach emphasized 
^ individualization techniques and made extensive use of paid aides 
for the teacher, . 
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(SiniLhwPSL Rt»).',i()na I iM'jh) ; wht»L hc»r or not ntlior i-oinjiDiuMils (if llioso 

prDjoi'ts romaiiiecl, ilv(» iiackrige maLcrlnls o\'lci\ loll iiiLo disimi^ in' 

ihv I' 1 ass rooni . ' 

« 

Aside rroniLliis rosiM* vn I i on , there is an im[)orLanL imp 1 i cat idii 
o{ tiuv, r ind ini;- that tiu» hyvorof funding — (ir 'the. si/^o of the "sec^d"- 
did not have ;i s i jL»n i f i eant effect either projeet i mp 1 einen ta t L(in 
or iMMit.lniiat ion in the i:lassroomr Not the wiowit- of funds, huf what 
thr Icu-al st\'iff '.t with these funds, significantly affects project 
(Miteoint\s. 

To put this finding into perspective, three qualifications niu^iL 
be remembered. First, the measures of continuation analyzed in this 
chapter reflect teachers' choices about using project methods or 
project materials, «uid th.ose choices may or may not accord with the 
district's decision on continuation. Teachers iray elect to continue 
part or al ! ol proje:t strategies with or '/itluuit district support. 
Thus, even though a shift in district economic fortunes may have 
forced district officials to revise their original expectations con- 
cerning Title 111 project continuation (as the recent -compla ints of 
local administrators would suggest), teacher con t inua t ipa deci s ions 
are not necessarily coincident. 

Second, it could be that the level of funding had "^indirect" 
effects on project outcomes that are not revealed in the i analysis . 
For example, the ava ilab i 1 i ty ' of more money might have co.nsiatenilu 
resulted in a set of design choices for an adopted project and 
these choices, in turn, might have had major implications for imple- 
mentation and continuation. However, the evidence does not warrant 
an inference of any powerful indirect effects.^ Higher fi'inding 

^The low partial correlations between level of fundjing and the 
dependent variables controlling for the independent variables might 
be due to the m.ul t icol 1 ineari ty be tween f und ing and tlie l:^ndependent 
variables. However, the percentage of variation in leve^'l) of funding 
explained by the other variables is only 26 percent; mofjt of tha^ ; 
variation is accounted for bv. the factors d isc-iis'sed aboVe . More-, 
over, the hignest zero-order correlation between any of/ the dependiMU. 
variables and level of funding is only .11. ^ee Table \l of Appen-^ 
dix A . • f . • . \ 
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levels are somewhat ^-sjociated with project characteristics. For 
example, the larger Title III grants tended, not surprisitigly , to 
be awarded to larger districts, to fund projects in a greater number 
of schools , or. to fund projects that spent more money per student. 
The larger grants also tended to support a particular educational 
method—individualized techniques. In this case, federal funds 
typically were used to hire the teacher aides necessary to individ- 
ualizing classjoom activities. Aside from these relationships, how- 
ever, the level of funding was not strongly related to other factors. 
Thus, it would appear — with the exception of individualization tech- 
niques — that "more" money does not necessarily purchase those things 
that matter; e.g., it does hot "buy" more or better staff training,' 
more ambitious projects in terms of scope of change, mote effective 
project directors, and so on. The low association between funding 
and other factors thus reinforces the conclusion drawn from the 
•evident low correlation betw^een funding levels and project outcomes. 
Namely, local decisions concerning resource allocation and imple- 
mentation . activities- — not amount of funding available — are prime 
determinants of the effect of the federal investment . 

Third, the iforegoing should not be' interpreted as meaning that 
"money doesn't matter." Our sample consists only of projects that 
received federal grants; although systematic analysis "of the effects 
of the presence or absence of federal funds is thus precluded, our 
fieldwork strongly suggeats that special outside funding does matter 
to local districts in a number of ways. For one, federal seed jnoney 
clearly allows districts jto undertake activities that staff are 
anxious to pursue but. that cannot be supported out of district funds. 
For example. Title III funds have been used to expand and refine 
teacher-initiated pilot projects for more general incorporation in 
district operations. Federal grants have also served as "start-up" 
funds for new teaching practices, many of which require substantial 
initial investment in hardware or staff development. In short , many 
local projects would not have gotten off the ground without Title III 
funding. Federal funding also bestows legitimacy on a local project 
and gives it the aura of "special status," which can serve at least 
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two functions: one, a "Hawthorne efffect" tha:: promotes enthusiasm 
for trying something new, and two, some measure of "protection" for 
politically controversial or peda'gogicailv untested educational 
strategies. 

Federal money, then, does^ matter to local da stricts, but whether 
* it ultimately "maki^s a ■ difference" depends . on how districts use it. 
(Chapter 6 returns to these issues in its discussion of school - 
district decisions.) ; 

PROJECT CHARACTERISTiqS V . . . 

The initiation of an innovation produces an adopted project, 
consisting of. a sl^ries of decisions about what is to be done and 
how to do it. We find it convenient to divide project character- 
istics into its educational method (also called the project treat- 
ment or instructional technology) , its scope of proposed change, 
and its implementation strategies. This section's main task is to ' 
analyze the effects "of these characteristics on the proj ec t * s -out- 
comes and continuation. , 

v:. The project^s initiation also;results in a second "product": 
The central office staff, principals, and teachers develop attitudes 
toward the project that seem to be crucial for its eventual outcome. . 
.,^hese attitudes are difficult to measure, part^ularly several years ( 
after the project began. They are nonetheless so important that we 
will briefly explore some of their effects before analyzing the 
effects of other, more measurable, project characteristics. 

■ . . ■ '' ■ . > 

Attitudes Formed During Mobilization ' 

A school may take up a federally funded . innova t ion because it ' 

chooses to or because the .district "asked" it to. How does the 

school'^ s and the. staff *s original involvement in the innovation 

affect project outcomes? 
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Teachers and project directors also report that , the con^trac- 
tual nature of the grant award is an incentive for staff to "keep 
trying" when . under other conditions they would give up or not put 
forth the extra effort. ^ 
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Of the' projects in our sample, 20 percent were originated by 
the teaching staff at a school. More typically, district officiaUs 
decided on a project and then asked schools' to participate in it 
(54 percent of the schools in our sample were chosen in this way; 
16 percent of the schools were part of a project that was imple- 
mented in all schools in the district; and 20 percent of the schools 
ac t ively sought to be in the pro j ec t ) . We found that schoo Is tha t 
had originated the project tended to have a more than average like- 
lihood of effectively implementing it; this positive effect was not 
ve.ry strong, however, nor was it significantly different from schools 
that, according to tlielr principals, had actively sought participa- 
cion in the project. Moreover, cont in^iat ion and teacher change were 
not significantly different in schools that had originated the pro- 
ject from what they were at schoolj that had been asked to partici- 
pate. Thus, the aggregate of school level evidence suggests that 
"who" originated a. project does not matter for its outcomes. 

This finding is surprising. One would think that originating 
schools would be so' highly motivated *that Lhey would produce better 
results than those in projects started by district officials. They., 
did not, for two reasons. . On the one hand, some originating schools 
did not seem to generate support for the project from the central 
office. Onthe other hand, some ' inhovatipns begun by the special 
project staff in the district overcame the initial skepticism and 
resistance of principals and teacher^. In short, the origit\ of a 
project seems to matter less than the institutional support mobilized 
during, the initial planning period. Let us review the patterns of 
mobil ligation that we described in our '^arlier research. 

Phase T of this study found that federally funded projects 
differed dramatically In/the seriousness with .which they were initi- 
ated: Many were started essientially to. take advantage of federal 
dollars, not to cope with a problem or need of. central, importance * 
to the district. Such opportunistic projects usual ly f.ai led to be 

l^Wre these variables- are entered in the regressions of 
Table 5.1, their effects are not significantOn^iny of the depend-g^ 
ent variables. 
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implemented effectively and tended to', disappear with the end of 
.federal funds; most' such projects, were removed from our survey 
sample. 

Even when the district's intent was serious, the way in which 
project support was generated in the school system varied consider- 
ably from district to district. Sometimes project planning .took 
place at the central office level without involving school building 
staff — particularly staff who were selected to implement the project 
Such *'top-down" planning, even with the best intentions, typically 
met with indifference or resistance from the school staff. One 
teacher made the revealing comment that, "This project hasn't warked 
out and its main effectf has been to cause * a close , well-organized ' 
faculty to turn to distrust each other. This was the result of 
forcing a program on a school, using an outside coordinator unfamil- 
iar with the school and faculty, and not having the full support of 
teachers. I personally felt the project ideas were good and could 
have worked if the teachers in our school had be'^n involved in the 
planning." Or, as another teacher in an unsuccessful project com- 
. plained, "The nroject was planned and designed without the knowledge 
and consent of the teachers at the school . . , the planner had 
hardly ever been to our' school . " Conversely, some projects were 
conceived and planned at the "grass roots" with only cursory review 
by the district federal program office. This style of project plan- 
ning also failed .to generate the necessary broad-based institutional 
support. As we have noted elsewhere, central office support is- 
cruci'$Ll, especially during the demanding first year or so 'of imple- 
mentation. As one respondent put it, "The super^intendent . had better 
believe in the project- — give his personal backing and support. 
Teacher confidence is essential; teachers should see in the begin- 
ning that top administration Ijelieve [in the project] and are . 
committed to it." . • " 



y .1 

In our sample, the correlation between the principal's per- 
ception of the extent of district support for the project and 
whether the project was originated or had been actively sought by. 
the school, was .05 and -.14, respectively. 
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Finally, we observed, in several case's, a pattern of early 
« district planning that we call full mobilization. * Tbese projects 
actively engaged the staff from thie preproposal period^ to imple- 
mentation, and gained consensus and commit ment from teachers and 
principals. 

The survey could not reliably measure these patterns of mobl- 
.lization. Instead^ to explore the effects that early motivations 
might have for project outcomes, we asked principals to estimate 
.the proportion of project teachers whQ'felt strongly committed to 
the project in the f i r:st yea r. Their answers provide a crude 
measure of teachers' overall initial commitment .•"^'^ This measure 
had a positive relationship^-and indeed consistently had the highest 
relationship of all factors at the school level — to the principal's 
assessment of various project outcomes (e.g., percentage of goals 
achieved*, change in teachers, and change in school organizational ' 
patterns') . •■ Moreover , the principal * s * assessment had a marked posi- 
tive correlation with the teachers' perception of the'percent of 
project goaJy achieved (.23), student, change (.20), teacher change 



The question was: (Initial School Commitment) In the -first 
year of the project ^ what- proportion of participating teachers would 
you say W3re highly committed to project ■ ideas and methods? 



All - of them *....:.... A3. 3% 

More than half ....... .29.4% 

About half . 13.6% 

Less than half . IA.7% 

None 0% 

Number of responses = * 14A 



Overall commitment tended to drop offi^between the beginning 
of the project and three cwr four years later after the end of federal 
funding. In part' this may be due to the project being ''unsuccess- 
ful," in part to a "burn-out'! phenomenon . The specific question was: 
(Present School Commitment) What proportion of teachers now using 
project methods '.are highly committed to the .approaches and ideas of 
the project? 



All of them 28.4% 

More than half . ; ' . 29.4% 

About half ........ . 17.4% 

Less than half ....... 12.3% 

None 2.6% 

Number of responses = 155 
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. (.22),. and continuation of the project's methods (.23). In short, 

12 

.the commitment of teachers has a pervasive effect. This commit- 
ment is generated, we believe, during the mobilization phase; thus 
in terms of implementation and project outcomes, the level at which* 
a project vas originated (school or district) matters less than the 
institutional support generated before implementation begins. 

Because teacher commitment is not a direct policy-input vari- 
able, it is not included in the factors of Tables 5.1 to 5.3. • It 
is pertinent to ask, however, what • strategies (i.e., policy-input 
variables) can be used to "generate commitment. The data do not 
focus directly on this question but we. can explore some hypothese^* 
A. number of piractitioners or planners of innovative projects, per- 
haps turned somewhat, cynical by a parade of disappointing change 
ef forts ,^ have come to believe that commitment is a well-nigh 
"immutable" ^eacher characteristic: . some teachers want to change 
and learn new practices , and some simply do not. We take a more 
hopeful view. Our field experience suggests that participation in 
the planning process— though not necessarily at the beginning — does 
enhance trenchers' commitment. (Our survey data show a .22 correla- 
tion between staff participation in project deicisions and commitment, 
though causes and effects are difficult to sort out- in this relation- 
ship.) For our sample, commitment is not strongly related to such 
teacher characteristics as sense of efficacy (.04), teacher experi- 
ence (-.02), and verbal ability ( .07) ; but it is related to. a varl- 

-ety of school characteristics, such as whether the school is under- 

ft 

taking other major innovations (.14) and the extent to which teachers 
feel that their school is a good school' (. 24) . This evidence 



When the overall initial commitment 'of teachers at a school 
is used as an independent variable in the regVessions of Table 5.1, 
it does not have significant effects. The reason, we* believe, is 
that it enhances teacher participation in project decisions (zeror- 
order correlation is .22) and. working relationships on the project 
(correlation is .26), both of which affect project outcomes. In • 
short, the initial commitment of teachers aff ects. the process by 
which the project is implemented and thus affects project outcomes 
indirectly. ' . 

102 



-82- 



suggests Llfat commitment can be generated and Is not a fixed teacher 
t ra i t . . 

The project characteristic most closely associated with teacher 
commitment is the scope of proposed cliange. The more effort the 

0 

project required of teachers, the higher was the proportion of com- 
mitted teachers (.14);. projects requiring an overall change in 
teaching style w.e re more likely to have a higher p ropo rt ion ' o f com- 
mitted teachers (.15). Though it is possible to interpret these 
data in\several ways, our fieldwork suggests the hypothesis that 
complex and ambitious innovations are more likely to elicit tlie 

ent hus i asm . o f teachers than rou t i ne " p ro j ec t s ^ Teachers seem to rise 

1 3 • ■ 

to chai Lenges . . ■ . " 

Another way to couch these inferences about commitment is in 
.terms of professionalism; that is, a primary motivation for teachers 
to undertake the extra work and . disruption of i\t tempting change is 
the belief that they will become "better" teachers and their stu- 
dents will benefit. Our strong impression, gathered, from field 
observation and interviews with practitioners at. all levels of 
school districts, is that convincing teachers of the educational 
promise of an innovation and of the opportunity for their prdfes- 
sipnal growth is an important factor in generating their support 
for a project. ^ . . 

In contrast . to • these essentially ".intrinsic" motivations, the 
literature speaks of using external rewards to motivate teachers to' 
change . We could not consider this. issue comprehensively , but we 
did examine the effects of extra' pay for training. Tables 5.1 to 
•5.3 include '\StaJ'f Paid for Tr'ainimf as one of the project's imple- 
mentatioh strategies. Sometimes projects use this incentive strat- 
egy to "get teachers to go along" with a project, or t o !'sweeteri 
the pill." In our sample, however, project teachers'who received 



It is interesting that specificity of goals and commitment 
are p5s i t ive ly related (.21)» Cause and effect are hard to' separate 
.in this relationship, but the following hypothesis seems plausible: 
Making the goals of the innovation clear ^ specific, and understand- 
able during mobilization enhances commitment. " 
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extra pay for training (about 60 percent of the sample) were no more 
likely than others to report a-.h^^^b^^^rcentage of project goals 
achieved or a change in their cXaiisrbom practices. Nor was pay for 
training relclted to improved s tudent. performance, or to continuation 
of project methods or materi<lls. In particular, Table 5.1 shows 
that the • regression coe f f icientvS for Staff Paid for Training were 
negative or insignificant. This finding lends support to the hypoth- 
t>sis that , especially for the small amount of financial incentives 
involved, intrinsic professional rewards are more important in 
eliciting effective teacher participation. A number of project 
directors have ffffirmed that although teachers appreciate the extra 
pay, the pay f alone cannot induce teachers to work hard to implemenf 

i 

new strategii?s if professional motivation is absent . 

In sumn!ary, these ^ata suggest that mobilization strategies 
can generate staff commitment, and therefore should be a prj^e con- 
sideration in project' planning. Project adoption, does not auto- 
.matically inspire enthusiasm. Teachers and administrators should 
be convinced of the project's educational promise and scope, and 
should jointly participate in its planning. As subsequent sections 
will discuss, institutional support of this nature is critical, to 
project implementation and is more difficult to generate once the 
project is under way. 

With the above background in mind, we now turn to. the analysis 
of project charact^2ris tics that result from project adoption. 



Though we did not ask about release time for teachers in^ the 
survey,, our field experience suggests that release time is a prefer- 
able strategy to extra pay 'for training after school hours. There 
are at least two reasons why release time can contribute signifi- , 
cantly to teachers' willingness and ability to change. One, the 
psychic and physical energy demanded by a full day of teaching makes 
learning hard to accomplish during after-school, evening, or weekend 
hours. Secondly, the provision of release time provides an impor- 
tant signal to project teachers concerning district commitment to 
the innovation. More than one teacher has commented: "If they 
(the central office) really cared about the project, they would 
give us sufficient time to learn what we need to know to implement 
it." Teachers generally are unwilling to take a project seriously 
if they believe t.he district does not. 

^ ' 104 



Blducatlonal Methods 

School people rare].y adopt, without change, nn innovation from 
butside their district, ^^/hether they wish to "replicate" a specific 
project they saw or heard about elsewhere, "or whether they want to 
apply a general educational concept; such as differentiated staffing, 
project designers tend at the beginning (i.e., during the mobiliza- 
tion phase) to adapt the innovation to the' local setting as well as 
to their own interests. For example, a mastery learning project * 
that was successful, elsewhere may, use materials that are not appro- 
priate for the staff, or students of the adopting district. Or, 
district staff may like the reading instruction strategies of one 
project, but prefer to use the classroom organization methods of 
another. As a consequence, the project adopted often comprises an 
amalgam of educational techniques and strategies^ as Chap. 3 des- 
cribed, that may be virtually unique to its district. 

Nonetheless, change agent projects also have cert^^in central 
characteristics or foci. Chapter 3 showed that any one of the tech- 
niques used by a project, in our sample can be identified empirically 
with, one of the following general types of educational approach : 
individualization (or student-centeted) techniques; classroom organ- 
ization change; curriculum revisions; community involvement ; and use 
of specialists for student needs. For the sake of trying to portray 
the distinctive character of one..project 's educational treatment as 

compared with others. Chap. 3 assumed that each project in our sample 

<? 

could have included all of the above general techniques to varying 
extents. Each project was thereupon given a score denoting the 
extent to which it concentrated on individualization, a score for 
its concentration on classroom organization, and so'on. The quanti- 
tative analyses of Tables 5.1 ta> 5/3 use these scores as measures of 
the projects' educational methods or instructional approaches. 

The quantitative analysis explored two questions. First , to 
what extent did the educational method of a project influence Its 
implementation, its effects on teachers and students, and its con- 
tinuation? The answer for our sample is that it did. to some* 
extent, but not very much. Secondly, did some educational inethods' 
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have, more significant effects on project outcomes than others?. The 
answer Is that they did, but the differences are not great. 

In particular, because none of the regression coefficients of*-*^ 
educational methods shown in Tables 5.1 to 5.3 are very large', it 
does not seem likely that educational methods (either individually 
or as a group) strongly affected implementation , effect , or continu- 
ation. This conclusion holds even under more rigorous -testing.. For 
example, a better test of the overall effect of educational methods 
is to assess directly how much of the variation in outoomes was due 

to educational methods alone. This test can be done in several ways. 
2 

For example, R , the percentage of the variation explained in the 

dependent variables, ranges in Table ^5.1 from a high of .50 to a 

J.OW of ,30, When the educational methcJci factors, are not included 

among the independent variables , the same regressions show a small 
^ 2 ' 

drop in R for each dependent variable, with the maximuin loss in 

V o 15 

variation being ,02, a significant ' but not very large effect. 

Thus, the educational method factors add little to the explanation 
of project impact and continuation. Another test is to use the 
educational ^method factors as the only independent variables. When 
the. dependent variables are considered to be a function of the edu- 
cational methods alone, the percentage of variation explained is 
.07, ,05, .08., and .06 for percent project goals achieved, total 
teacher change, total student change, continuation of project meth- 
ods', and continuation of project materials, respectively. Thus, 
even when, the confounding effects of other variables are removed, 
the choice of educational treatment had only a small effect on 
project outcomes. 

"^^cause the regressions using, the educational method factors 

2 " 

as the only independent variables appear to produce an R that is 
greater than R"^ obtained by "adding" the educational method factors 
to the other independent variables of Table 5.1, we suspect that 
educat ional methods did have some indirect e f f ec t on proj ec t outcomes 



The appropriate statistical .tests for most of the statements 
made in this subsection about educational methods are F-tests, the 
details for v{hich are not supplied in this report. 
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That is, the choice of trocatment was assoclntod with ottier InipJe-,, 
mentation choices that affected how the project turned out* For 
example, projects concentrating on community involvement were some- 
what mere likely to require extra effort from teachers, to Involve 
considerable training activities, to use consultants and meetings, 
to have teacher participation in project decisions, and to engage 
tlie* principal ' r, cupf^ort. Some of these local choices affected 
project outcomes, as we will discuss. As another example, Indi- 
vidualized student-centered projects tended to be funded more heav- 
ily, to be more specific about project goals, to rely more on skill- 
specific training, to have less (participation by teachers, and to 
have less .'involvement by the principal, . Through these associations, 
the treatment exerted its indirect influence on outcomes. 

Some direct e f fee ts of educational method also can be seen in 
the tables. Projects concentrating on curriculum revision had a , 
slight tendency to be implemented more effectively and to improve 
student achievement more th^n did other methods. Classroom organi- 
zation projects were somewhat more likely to result in teacher 
change and to have project methods — not materials — continued bv 
teachers , ^ 

This finding is not surprising in light of the distinct foci 
and activities of these educational strategies- That is., projects 
emphasizing curriculum development and revision typically begin with 
well-specified objectives and focus speci f ically on student cogni- 
tive growth. Classroom organization projects do not explicitly • 



More precisely, the appropriate F-tests showed, that the differ- 
ence in percent project goals achieved between projects focusing on 
curriculum revision and those focusing on community involvement or 
the use of specialists,,^^ significant at the .05 level; the differ- 
ence between projects focu^^ng on curriculum revision and thosie 
focusing on individuallzat iolj or classroom organization was barely 
significant at the ,15 levelJ Similar results held for total stu- 
dent ^ improvement , l^-tests. aii5o indicated that, for projects focus*^, 
ing on classroom org&ft4=3=aiffon , the coefficients for teacher change 
and continuation of project methods were' significantly different (at 
the .10 level) from other coefficients (with the exception of that 
for community involvement). 



ndcfress st luleni: achlovumtnit , hut conctmt: rato on rhnngln^ t rad ! t Ion;i 1 
patt<»rns of V 1 assroorn InanaR^WMt: and studonl /tcachcr lnt:tM"acl Ion. 
Consociuoiu ly , projects of this nature rec|(iirc the most s I cn I f I can t 
cliangos in tho way teachers do th lnj;s-~rhangos In method that, once 
assimilated, are likely to b*^ sustained, 

Tal'lc 5.1 shows that projects deslRiie'd to have a great' deal of 
community involvement appeared ' to have a d i rec t ' e f f ec t on promoting 
teacher change, similar to the effect of classroom organization 
projects. (That is, the regression coefficients for both community 
involvement and c 1 ass r6om' organiza t Ion change were significant and 
nor much different from one another.) However, community involve- 
mtMU did not significantly affect continuation of project methods; 
the apparent reasons are the two very different, though overlapping, 
ways the projects sought to involve the community. Some strove for 
, direct parent, involvement as a major^part of their activity; others 
centered on using community resources or undertaking field trips and 
inyited parents to observe and assist. 

We found that pro-jects having direct parent involvement as n 
maj^r goal produced teacher change and were likely to be continued 
by teachers after the end of federal^ funding— p robab ly , we suspect, 
because p roj e^ct^taf f had a fairly high level of initial commitment 
to notions of parent participation in school affairs and governance. 
The other type of cqmmunity involvement projec ts had a slight nega- 
tive but not significant effect on continuation. Field trips are 
taken more casually; they are typicality disc;^rded , as an ''unessential 
activity" when soft money goes away. Although Phase I of our 
research did not deal speci f ical ly with parent involvement, this 
finding deserves the more detailed exploration given later in 
Appendix C. j _ 

In summary, the evidence of both Phase! and Phase 11 supports 
the broad hypothesis that the educational method chosen determines 
a project's implementation, effect, and continuation to only a small 
and limited e;ctent; and much of this, effect comes from . implementation 
choices m^de during mobilization that are associated with particular • 
types of treatments. Within this general finding,, however, it does 
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appear that some types of treatmen^ts ha\a greater incremental impacts 
than others, controlling for implementation choices. Specifically, 
projects concentrating on curriculum revision were marginally more 
likely to achieve their goals and to increase student achievement. 
Classroom organization projects, as well as projects focusing on 
parent involvement, were more Idkely to produce teacher change and 
to have their methods, not their materials, continued by classroom 
teachers after the end of federal funding. On balance, however, an 
educational treatment did not automatically fulfill its expectations: 
Wnat it was mattered le$s than how it was done. 

i f 

Scope of Change 

A project design fac'tor of considerable interest to policy-., 
makers and practitioner^ is the scope of cj^ange proposed by an inno- 
vative project — that is, the amount and complexity of change required 
of teachers in their classroom practice and behavior. A number of 
policymakers ,* especially ;thos^^^ in -the grants revie^yprocess", 

have expressed conc'^'rn that proj ects attempting to make ambitib~us~ 
and comprehensive changes will run into .serious difficulties, and may 
produce no'" change at all. In that case., projects with fewer and 
.narrower objectives would be "safer bets" for federal or state 
investment. Our Phase I analysis found that the pdlicymakers ' con- 
cern is only partly justified. Ambitious projects often were less 
successful in absolute terms of the percent of project goals 
achieved but they typically stimulated more teacher change than 
did projects attempting less. Teachers^ evidently welcomed the 
challenge. ' ' . . 

Requirements for Change and Extra Effort. . For the Phase II 
research, we felt that the vazious^ dimensions of "ambitiousness" 
and "complexity" had to be unpacked. We knew, first of all, that 
projects differed in the nature and extent of change they sought to 
bring about. Accordingly, the survey asked teachers to indicate . 
the type of change required of them 30, percent responded that the 

"^^The specific question and distribution of the 1016 ^responses 
were: (Overall Change ^'Required in 'Teaching) Innovative projects^ 
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project tried to change their overall teaching style, and 14 percent 

that no change was sought. 

This variation can be explained to some extent by the substance 

•of the innovations^ some of which, by definition, assumed. more change 

in classroom practices than others. For example, projects that relied 

heavily on tKe use of specialists typically asked. little more from 

the teacher than help in identifying children who needed specialists' 

attention. The zero order correlation of (-.09) . between this educa- 

■. tional method and overall teacher change required, suggests that many 

'of the 14 percent of the teachers answering that the project did not 

try to change their teaching style were "j?nvolved in projects of this 
18*^ 

nature. Similarly, the zero order correlationsr(.15) , (.12), and 
(.11) between the overall change required and the educational methods 
of community invoJvement , classroom organization, and curriculum 
revision are not surprising. Projects emphasizing thase educational ^ 
methods can be expected to assume comprehensive change in teacher 
practices by virtue of the fact that they require significant depar-r 
-~i:ture.:::f rom- t ra di tional methodologies and behavior. Nonetheless, the 

zero order correlations . are low enough to indi'iEaTeT'tlia' L uv era-l:3rr — - - 

change required cannot be taken as a proxy for project' type or edu- 
cational method. ' The data suggest that overall • change should be 
viewed as a local project design choice that is relatively/ independ- 
ent from choices concerning educa.tipnal method. 

» ^ ... • 

The same conclusion holdsf for a second^ dimension of project 

: / ^ 19 

scope: the extra ef fort , required of teachers. . Again, the projects 

vary in' terms of the scope of change they try to bring about. Would 

you say this project . 

, • Tried to change overall teaching style . . 30% 

Tried to change specific teaching techniques .... 56% 
Did not try to change teaching style or techniques . . 14% 
18 

The analysis divided the. question on the scope, of teacher '\ 
• change into two parts: overall teache:r change required and* not 
required. This two-valued (or dummy) va'riable is the one .used in 
the ccrrelations and ..regressions . 
19 

Thevspecific question and distribution of the 1032 responses 
were: (Extra Effort Required of Staff) "How much extra effort did 
this project require of you during the first year you'^ implemented 

110 ■ ■ 
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variecl" considernbly ; moreover, projects could call for considerable 
effort with or without also requiring an overall change in teaching 
(thus, the correlation between these" two variables is a low ,15). 
For example, projects focusing on community involvement A^ere. likely 
to require both extra effort and an overall change in teaching style; 
individualization projects tended to focus more narrowly on specific 
areas such as mastery learning techniques or remedial reading, but 
could require a high level of extra effort from teache.rs. 

Thus, the two dimensions — overall change in teaching and exTra 
effort required — vary independently of each' other. Moreover, school 
officials and project directors, on similar types gf innovations, 
varied in the amb^itiousnesB of the requirenients they imposed on 
t lie ir staffs. . 

y • 

How does the requirement for' change affect proj ect impl ementa- . 
tion^and continuation? The data in Table 5.1 indicate that projects 
requiring overall teacher change had the same effect on the variables, 
of percent gbal.s achieved' and improved student performance as did 
other projects (the regression coefficients are not significant). 
Contrary to the fears of program grant officers, projects attempting 
^g^han^.e in o^^^^^ style were no more or less rikelyf-to fail 

(or to succeed) in meeting TheiF db je^Fi^^ 

cliange than, were more rtarrowly focused or less ambitious projects. 
In other words, attempting less does not necessarily assure greater" 
success, V ■ . 

Thew)verall change required did, however, have a' major positive 
effect on total beacher change (.16).. Furthermore, as Table 5;2 
indicates, projects that required overall change apparently promoted 
change in all the dimensions of teacher behavior measured in our 
analysis: the teaching o.f basic skills (.10); the. ability of teach- 
ers to recognize and attend to individual students' 'special learning 
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. problems (.11); the way teachers organize the-ir classrooms (.16); 

and the way teachers work with students in affe.ctive development " 

(.15). The findings of Phase II, then, strongly support our find- 

• ing.s in Phase I: The more change that is required of teachers, the. 

more change they will make in their teaching style and behavior. 

Similar results hold for the effects of requiring extra effort 

from teachers. Many seirious innovations, call for a considerable 

sacrifice by teachers, par t icula rly during the project's first year. 

The evidence inTable 5.1 suggests that this requirement per se did 

not hurt the project's implementation or its prospects for improving' 

student performance (the coefficients are -.OO-and -.00, respectively) 

Table 5.1 also shows that the more>that extra effort was required of ■ 

teachers, the more they changed (.17) and the more extensive wa^ con- 

9r) . 
tinuation of project methods (.13)." , ^ 

In. light of the strong effect on ^-eroher change, associated with 
ambitious projec^j^ it is somewhat surp^.sing that projects requiring 
an overall change in teaching style- did^; not . have a s t ronger in f luence 
on continuation of methods than the .07 indicated in Table 5.1. Our, 
field experience suggests several reasons for this "drop-of f . '/j; 

■ For. one, most of the twenty-four secondary schoo.l projects in 
our sample aimed at basic changes in. teaching patterns. - Such pro- ■ 
jects--by-:theiii_.\^e_„ry^na^^ to sustain after the encf of 

federal funding. For example, a number of these~lTiglvTchoo^ 
involved alternative schoals,;!^^^ Although such projects may represent 
an interesting social/educational experiment or a convenient response 
to parental pressure for educational options, they also*" wore typi- 
cally expensive and difficult to continue on district funds-r- 

20 . ' ' 

These effects are sharply diminished , 'as Tab le 5.3 shows, ' 
when the extent of teacher change caused by the project is intro- 
duced as a control variable in t\ie regressions. In other words., 
ambitious and demanding proj ec ts promote a changef in teaching style 
which-7-when it works— is continued by the teacher. In contrast, 
the positive effects of Extra Effort and of Overall Change Required 
on continuation of materials do not change much when the extent of 
teacher change is controlled (compare the last columns of Tables 5.1' 
and 5.3) . 

■ / ■ 
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particularly in. a time of declining student enrollments. The sec- 
ondary Ijevel "school within a school!.' projects also required special 
resources that were often beyond the capacity of many school district 
budgets. 

Similar problems occurred at the elementary level. Projects, 
calling for overall change in teaching style often employed support 
strategies that were difficult for a district to sustain after the 
end of federal funding. For example, the field trips rarely sur- 
vived, and the classroom aides that usually accompany classroom, 
^organization proj ects , such as open education, often could not be 
supported with district funds. Left with neither federal nor district 
funding, teachers often found it difficult to continue a project 
intact. Instead, they had to adapt project practices to conform, to 
budgetary realities, ' 

Another continuation problem for ambitious projects was a *'burn-.. 
out-* phenomenon. Bum-out seemed most prevalent among teac'liers who 
were active in project design and management affairs. Insofar as 
these teachers were given opportunities to continue to expand their 
experience with the project — for example, by participating in outside 
dissemination .activities — they seemed to fecoyer quickly from^ their 
post-project letdown, , ^ 

In summary,. our data do not confirm the fear that projects 
attempting a broad scope of change are likely to break dbwn and are ^ 
— likely_ to_be_di_sjrontin Despite a variety of implementation and 

continuation problems, projects requiring change in the~~overaTl — : 

teaching' style of project staff and calling' for extra effort result 
in significantly more teacher^ charige.' 

• "Specificity of Goals , Another surprisingly elusive dimension 
"of project scope is . the specificity of its goals. In attempting to 
measure that (dimension in Phase I, we ran into a recurrent diffi-. 
culty: Participants ^from different projects using similar: treat- 
ments — e,g, , using the same reading program — described their objec- 
tives in very different terms. That is, the same innovation could 
have very diffuse or very narrow goals depending on"' the perception 
of the project st^ff and, more particularly, on how mobilization of 



the project was handled. In Phase II , consequently, we decided it 

was more important to measure the teacher's perception of how sp.e- 

' aifia the goals of the project were, than to attempt to define the 

21 

■■inherent." breadth of an innovation. 

: Table 5.1; shows that specificity of goals had a major effect on 
implementation: The more sp'ecific the teachers felt the project 
goals were, the'higher the percentage of goals the project achieved 
(the regression coefficient is .17) . ,One reason for this strong 
effect is simply that some of the projects in our sample, particu- 
larly remedial projects, had a well-defined programmatic design and 
narrow instructional aims, It is. easier, all o^her things being 
equal,, to achfeve narrow gbals, and it is still easier if the. piro^ 
ject design 'is^ prograrmaim^calty .specif ia ' ^ . 

.This explanation of specificity makes sense for projects, or 
elements of projects, that can be programmed. For example, the 
development of project' materials can be carefully planned. Table 5.1 
suggests, that specificity, in this area promoted not^ only implementa- 
tion but also continuation of project materials (the regression coef- 
ficient is .15). Similarly, specificity on projects aiming toward 
cognitive development led to improvements in student achievement 
(see Table 5.2). However, many of the innovations in our sample 
could not be so programmed. What explains the effects of specificity 
in these .(?ases? ^ . 

Obr. f£eld^ experience suggests that when projects emphasize, 
change in classroom or ganlza tion--- change that-^iS: inherently amorphous 
-and^not susc^ptj^le^^ specification in a programmatic s^ense — teach- 
ers need clarity about project precepts. For exarmpi¥",~we^haveT s^^ 
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similar open education projects that were decidedly different in th 
extent to which the- staff grasped the philosophy and operational \ 
objectives of the innovation. Even after three years of operation, 
a lack of clarity still remained on some projects; teacher's . could 
not be expected to continue the methods of the. innovation in these 



cases . 



The quantitative evidence supports this interpretation of spec 
ificity. First of all, Table 5.1 shows that specificity had a posi- 
tive effect on continuation o f project methods (the regression coef 
ficient is Ml) ,. Secondly,, we created a subsarnple of projects . that 
heavily . focused- on individual ization , and a subsample of projects 
thaD; concentratfed on classroom organization changes.., Th(?.se sub- 
samples, which will bfe discussed in more detail later, allowed us 
■to compare the effects of ""such variables as specificity for the- two 
contrasting types of innovations, < Individuallzatibn ' pro jects we te 
more specific ,^-onth^ ave rage than classroom organiziation projects 
. which also varied considerably more 'in specif ici-ty , Moreover , 
regressions similair to Table 5.1 (but using the' two subsamples) 
show that when- clarity (specif icity) was^achieved on classroom 
^organization projiects, t?he , continuation of project methods greatly 
increased increases:.in specificity on individualization project^s 
did not significantly affect, the proj'e^cts* effect on continuation.. 

Specificity, then, can be seen; as having two components:, pro- 
grammatic spefcif ic.ity a factor that is fundamentally :3 project 
design issue, and conceptual clarity, an understanding. 'that must 
be achieved—du.r i-ng— the^ course o f ., p ro j ec t implemen ta t ion . _j:pnc_ep^tjjal 
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clarity — the extent to which staff are clear about what trhey are to 
do-rcannot he guaranteed by the use of packaged materials or lec- 
tures from. c(5n.sul Cants. Rather, it must be achieved through prac- 
tical, concrete training activities that permit project staff to 
understand the significance of project precepts and strategies for 
their own classrooms. Iri short, clarity is not something a staff 
.t:an. be "given" at =the outset, although it can be facilitated by 
well-specified programmat ic statements . For projects attempting 
comprehensive change, we suspect ?that a lack of clarity, rather thanr 
lack of programmatic specification, generates severe .Implementation 
difficulties and the disuse of the project after the £nd of federal 
funding. ' . 



In summary, this subsectJLon has examined" three aspec ts of the 
scope of change implicit in the design of innovative projects: the 
overall change ' requi red. .of teachers ,. the extra effort requTred of . 
them, and the specificity of the project's goals. The analysis 
suggests that (a) the more change asked of teachers , . the more they 
respond; (b) asking teachers tp put in extra, effort may create ;some-^ 
■problems in implement at ion ,' but . teacher change and continuation are 
more; likely , to occur with this kind of sacrifice; and ic.) teachers' 
can implement innovations better if they clearly understand the 
project^s. purposes and precepts.. - . 

Implement at ion Strategies o .. 

Among the most important products of the planning and mobiliza- 
tion activities That~~l e ad~t o~an~~a"d optre d^p r o j ect— are— implement at ion' 

strat^egles — a series of decisions and choices,' explicit or implicit,- 
on how to put the project into p'^ractice. Although the strategies 
chosen ar^ somewhat associated with the type of educational method 
adopted, local designers exercise considerable freedom in, their . 
decisions; for example, similar reading programs may he implemented 
quite differently among schools and school systems. Chapter 2 
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discussed the strategies we observed in the field; Tables 5.1 to 5.3 
show the effects on project outcomes of strategies we were able to 
measure in comparable ways for the range, of innovations in our 
sample. As a groups* these strategies had major effects on the imple- 
mentation, impact, and continuation of projects. Moreover, they 
differed in their conduciveness to particular outcomes. 

We~discuss below the findings for each strategy listed in the 
tables^ Two qualifications should be kept in mind as we. do so. 
First,* our list of strategies does not exhaust the variety of major 
choices made by local implemeritors . For example, some Strategies 
were so particular to an instructional approach • that they 'fcould .riot 
be generalized for oar analysis. Other, strategies — e.g., selection , 
of staff — were difficult to measure; we could not include them ' 
directly in' our statistical analysis but we examined them in our 
fieldwork. .Still other strategies, such as local materials develop- 
ment, were rneasured to some extent b^ut were so highly ^ interrelated ^ 
to other factors that they could not be included in the full; analysis 
shown by Tables. 5.1 and 5.3; -rather, we explored their effects in the 
partial analyses discussed througho'ut this chapter. ^ '' 

The reader als\) should be aware of the artificial separation of 
strategies imposed by the statistical analysis.. In reality, the 
overall strategy 'of each local project comprised a mix of strategic 
choices. Our statistical method ex^ines component strategies, not. 
the. strategy as ,a whole. Our conclusion synthesizes .the separatte 
results into a judgment about, what cor^stitutes an effective overall 
strategy. 

Allocation Decisions . Local planners must, also decide about 
the allocation of the project's resources. Table 5.1 lists two , 
pertinent' allocation choices: the number of project schools and 
the concentration' of the project's funds (i.e., the number of stu- 
dents served for the available .project f unding)~ FevTliistricts-make 
aHocation decisions in the purely objective manner of an R&D deci- 
sion.^o^del. .School officials rarely have an- "experimental plan" in 
mind that^can be stated in such terms as, "We will try the innova"- 
tion in scho^ls^ A and B; if it improves student performance, we will 



spread it to schools C and D." Instead, organizational and political 
considerations usually govern such decisions. • 

Fpr example, 20 percent of the projects ip our sample were orig- 
inated by the principal and staff in a school. The most "successful" 
were begun' several years before they received a Title III grant. 
To continue their efforts, they prepared a Title III proposal with 
the assistance of district personnel. But district officials often 
felt the need to be "equitable," particularly when "federal goodies" 
were invplved, as an assistant superintendent put it. Thus, because 
the award of a federal grant would bring "visibility" to the. project, 
school officials in two of our case studies added other schools to 
the original site; In one of these situations , ;one school was added , 
in each of the areas represented by school board "members . . • 

■■ In contrast, approximately 16 percent of the projects in our 
sample were distf'rict-wide projects ^ most of which involved curriculun^ 
rievision and individualization. Several of- these projects were pre- 
ceded by earlier experience with the innovation. Most, however,'', did 
not have an earlier:^'development" $it a local school; the districts 
were att'empting to install a new program district-wide without the 
step-by-step procedure often favored by planners. We do not know • 
ali the political and organizational elements that affected the^.e 
: decisions. It seems clear, however , that a number of districts 
started with an identified educational need/but "designed" a. resource 
allocation plan whose prime- concern was to obtain as large a grant 
as possible and to diatribute it as fa,T as the money would go. . 

In deciding on. site amd funding allocations, then, districts 
are influenced by o^ther concerns besides the desirable educational 
outcomes listed in Tables 5.1 to ^^.3. For^, the range of variation in 
our sample, however, neither the number of schools n^r the funding 

per student strongly affected project effect and continuation in most 

■ 23 

cases. . : , • . . e 



. The number of schpols for projects in our sampl4 ranged- from 
1* to 44, and the funding per student from a high of over $2000 to a 
low of about $12. The correlation between these variables is -.21; 
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The Gkceptions to this genetal finding are noteworthy. The 
greater the number of schools in a project, the higher was the^pro- 
portion of p-roject materials continued (the regression coefficient 
is .12 in Table '5 . 1) . This relationship results from prof.ects that 
fiir.riished new curriculum^ material or educational hardware that was^ 
adopted as districtwide policy. In such cases, it is clear that 
teachers comply with official district policy and use the materials 
pure: based wi th federal money . Yet these same pro j ec ts produced 
lictlo if any teacher change (the coefficients of teacher change in 
Tables 5.1 and 5. 2. are mostly negative but not signi f icant ) . More- 
over, tb*.* continued use of project materials was not always accojn- 
panied hy a continued use of proj ec t .^me thods (the coefficient in 
Tables 5.1' and 5.3 is positive hi;ii not significant). "Tn,/short, some 
districts seem to have used Title TJI."s^eed mo^iey" more tb:,-purchase 
up-tro-date materials and technologies than' to promote basic educa- 
tional reform. 

Th^ other signi f icarit' rela tionship shown bv the tables involves 
the, concentration of funding: . 'Hie higher the funding^per student, 
•the greater was the improvement in student -outcomes (the coefficient 
of total student change in Table 5 . 1 is . 09) This Effect comes 
primarily from remedial projects that focused on individualization, 
were located in. schools in areas of lower socioeconomic status (SES) , 
and sought to increase the perfo'rmance of students who were belpw- 
average achievers. The heavy concentration of fiinding for these 
remedia I ^proj ec ts paid for classroom aides. Of all the projects in 
our. sample, 77 percent hired aides or paraprof essional staff; 63 per- 
cent of these projects eithpr decreased (Sx eliminated their aides ^ 
after the end of federal funding. All proje*cts haying a heavy con- 
centration of, funding per pupil reduced their aides. The' teachers 
in these proj ec ts * indica ted tha t. they^ had not changied, their fetyles 
vei^y much, nor were they currently usingf project ifi^thods or materials 
extensively (the coefficients of, fundirtg per student for the' con- 
tinuation variables listed in Tables 5.1 j:o 5.3 were -not significant) . 

• ; : ^ '. : '• . . iJ. 

the correlation between funding level and number of schools is .17; 
the correlation be tween funding, level and funding per student is .18. 
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In short, theso remedial projects improved student performance, but 
we suspect this effect will be short-lived. Successive generations " 
of students are unlikely to benefit, and the federal money spent for 
aides will have had only A' fleeting effect on district practices. 

In summary, the Phase II sample omits some of the projects in 
the original sample that were inadequately funded to meet their 
purposes and therefore were discontinued. Within the range of vari- 
at*4on in our sample, we found no clear guide as to the "optimum" 
number q,f. project sites or funding per student.- These local alloca- 
tion decisions are usually the result of a complex blend of organi- 
zational pol i tica 1 , and educational criteria. Without being able 
to eontrol for these complexities, we cannot tell whether "better" 
decisions could have been made. In any event, the lack of strong., 
relationships suggests that these decisions ifnatter less for proj(?ct 
outcomes than do other local choices and conditions. 

Staff Training and Training Support Activities . The components 
of staff training wisre as\numefous and diverse as the educational-, 
s tj-ategies' themselves Np two projects appear to have^pursued the 
•same- design or raix of training ac'tivities, perhaps because no one 
has devised a theory or "model" of what an effective staff training 
program should look lik^' (or , similarly , an effective staff deVelop-"^ 
ment. or in-service program). The training designs actually pursued 
therefore are . largely "bes t guesses." In prac t i"ce7~some of thein 
were ins t rumen tal in- successful proj ect . implementation ; others turned 
out to be inadequate. v - 

-To shed light on what constitutes an effec tive staf f -.^training 
program, and how important it is to pro j.ec t success and continuation, 
the survey asked teachers to indicate the type, amount', and format 
of the training they received. InIs also collec ted'^imilar informa- 
tion on support activities related to training — e.g., the use of 

24 

Approximately 22 percent of the teachers surveyed mentioned 
"training inadequate" as a problem they had to deal with "during imple 
mehtat-iQTi^- In terms of frequency of teacher, response , inadequate 
training was "^he- fif th mos t cited ^ factor on the list of 39 implemen-j 
^taion problems. ' ^ ■ 
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consultants, staff meetings, and the assistance of resource personnel 
in the classroom. Table 5.4 lists these questions and the distri- > 
bution of responses. Our fieldwo'rk made it clear that for analytical 
purposes it is not enough merely to record, that certain training 
activities went on. The same activity may be irrelevant in one pro- 
ject and useful in another, depending on the kind of, project or the 
.way the activity was, carried out. For example, staff workshops may 
be useful in a program attempting to implement mastery learning 
techniques, but irrelevant to staff who are working to put a new 
student counseling program into practice. Or, outside consultants 
on one project may be able to provide concrete, useful information 
to teachers, while .on another they convey glib generalities that the 
teachers dismiss as a waste of time. Consequently, we also collected 
information on the quality of the training strategies. 

Because there is no consensus on what constitutes "quality*' 
trainiTig, we asked teachers to rate the utility of the training and 
support activities they received (see Table 5.4 for the specific 
questions)." This assessment of quality in practice, as opposed to * 
quality in the abstract (or, what is useful as opposed to what 
"should be" useful) was then analyzed in terpis of project outcome * 
factors and teacher continuation of project activities. The results 
are presented in Tables 5.1 to 5.3. The following discussion will 
also consider the relationship between the quantity and quality of 
tjiese variables and explore wh^t quality might consist of. 

The regression coefficients in Table 5.1 suggest that well- 
conducted ''staff training and staff support activi,ties improved pro- 
ject- implementation, promoted teach^^ change , and increased the 
continuation of project methods and materials. These variables 
alone (i.e., without * other factors such as the institutional setting) 
account for a substantial portion of the variation in. project effect • 
and continuation. From one perspective, of course, this is not a 
surprising finding. After all, teachers have to acquire the. new' 
skills or behavior if project-related changes are to influence stu- 
dent performance or be continued. Even so, the varying contributions 
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Tabic 5.. 4 

STAFF TRAINING AND SUPPORT QUESTIONS 

(Amount Pre training) About hou many hours of training seseipns ( lectures or tjorksh'ops) did you 
attend before you began implementing the project i^in your classroom? 

1 None . . 16Sr ' 

2 Eight or fewer hours (consider 8 hours one work, day) 20 

3 Nine to 24 hours (consider 24"hours three work days) . 21 

4 25 to 40 hours (consider 40 lioU^s one work week-') 20 ' 

5 More than 40 hours . i, 24 i 

. o 

■ „ No. of responses « 1000 

(Pretraining Usefolnfesj;^) Hou useful were these sessions for dealing uith the problems that 
arose during ^X^e .pi'cj'if^t? ' ■ .« ' " 

19% 19% 23% 21% 9% . 7% 2% : 

. L I : 1 , ! J I I o. 

No. of responses = 840 

■ -7 6 5 4 3 2 1 

Extremely Not at a.Il ' 

useful • ' useful .;' 

(Amount Rtrs-c-)L>9r Training) About hou many hours of training sessions (looturcs and uorkshops) ' 
.did. you^Li^ tend durituj the first year you uere actually implementing ' the project in if our classroom? / 

1 None . '. k . ; Hx ' • 

2 Eight or fewer hours ...... " 23 

3 Nine to 24 hours ^ ; 27 C. , . 

4 25 to 40 hours 17 . ' ' .! . . 

5 More than 40 hours i*. ; ^..i 19 .'\ , .' ' 

No. of responses » 991 

(First-Year Training Usefulness) How .liseful uere these sessions for dealing loith the problems' that 
ax*ose during the project? / ' ' 

16% 22%.. 25% . 19% 10% 5% 2% 

I 1 — : — I 1 I < I 

■ . . ■ No. of responses = 836 ' 

.- -r.^:^^^ : 7 6 5 4 3 ..; 2 1 

4'Exy;eraely ' Not at all 

. ' '■ useful ■ '. useful 

(Amount Training Later) About hou many hours of training sessions (lec^tures or workshops) did you 
attend' a fterr the first year^^ou implemented the project? • " ~. 

^ ,^ 1 None '. 26% 

'"'"[i;.'^ 2 Eight or'^fewer hours 27 

^ 3 *Nine to 24 hours 22 

4 25 to 4.0 hours ..-r.... 13 

5 More than 40, hours ;^ ."V. ; . . . 12 

^ . No. of responsqs - 9'74 

(Later training Usefulness) HoU useful Dere 'these sessions for dealing with the problems that arose 
durivjg the project? '' 

13X 25% 24% 20% 12% 4% 3% 
• . 1 I I t I > I 

f No., of responses » "732 

. r 6 5 4 -3. ' 2 - l ■ . 

Extremely Not *t..all- " ' • 

useful useful 

(Amount Classroom Observation) About how much time did you spend observing, the project's methods in 
other teachers' classrooms before you began implementing the project in your classroom? 

1 None ... 65% 

' ■ 2 ' Half a day or less % , .' . . 15 

3 One to two days : ' 12 

4 Three days or more 8 . 



'No; of responses « 993 

1.22 , ' 
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Table 5.4— Continued ■ . 

(Observation of 0^her Classrooms) Hop uoeful iJao this expei'icnac fov dcalifuj u^Un tlu: tfrohU:nui^^: ♦ 
thitt ui'oat: duruii] t:he project? * 

17% 19% 25% 192 12% 63; 17. 

^ ^ ' — ' =^ ' — ' No. of responses ^ 3.'»^ - 

7 "^'t). 5 4 3 2 1 

Extremely Not at all * 

■ useful useful 

CAmount Classroom Assistance) Uuriruj the fivG.t year you actual ty implement^ projecft t eahnii{iws 
in your alaoaj'oom, about >icu many times did t/c;j< receive aGcintatici: or advice fvom rrjouyco i>dt*c.(^}ir^ 
di' project staff -jho cane into your alai^nvoom to obaerV'e atui rnuke :yug>jei)tio>uyy 

I None r. 

^,1 or, .2 times •.: 1« 

J 3 to 5 times 24 . 

4 6 to 9 t inen ,-....< 16 

■ ' 5 10 to 15 tin...s : l'» 

6 More than'^5 times : II 

< No. of responses = 98'> 

(Classroom Assistance) Ho!J uoefUt uere thcoc visits fov deaLinj '.u'th Ow. prnhL't'ir, fh::(. aro::e dur- 
iftg the projeat? ' ■ 

16% 19% 22% W% 11%" 10%' : JZ * 

^ ^ ' ^ ' ^ ^ No. of rt.-sponst's = 808 

■ 7 ft 5 3 ■ 2 • 1 

Extremely • Not at all ■ 

useful useful 

(Amount Consultants) Hou many times did you veocive craintni ^idvicc or d('ri,jn:Uri:H\^ni; fr^iri can- 
sultayits from outside the district as part of the yrojaiM? 

1 None '^ V" 

2 Alrost none ;* ■ ^ ^ 

3 A few times . 30 

4 . Quite a few times ; 17 

5 Many times 11 

No. of responses = 1001 

(Consultants) Hop useful Dere these consultants to >;ou^'hi iraplcm^inting- the projent'/. 
11% 19% ^,27% 18% 10% 11% 4% . ■ 

' ' * ^ r-^— ' No. of responses = 724 

?' 6 5 4 3 2 1 

Extremely - Not at all 

■ useful . ' useful « 

.(Amount Project Meetings) H(>u\ often did you at teyui riccti>{(js i^ith other project tiuicheri't lo 
discuss the ideas of the project and lioid heat to cMvc problcns that mt(jht ■ ocour? 

1 Never 14% 

2 Only a few times a .year ••. 28 

3 Every couple bf moirths ". • : • • • • 

4 About once a m'V . !i : 1^ 

.5 Several, times % month 

^ „ , No. of^ responses - 9H3 

(Project ,Meet.in«s) Hou useful ucre these nee ting a for dealuuj oith\the prooh'na thr,t arose// 

14% 22% <;26% 20% -7% ."2%- 

^ * * ' ^ ^ ' 1 No. of response's = 85H 

7 6 5 4 3 2 I . 

Extremely Not at al I ' 

useful ' r ' useful 
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made bv training activities can furnish useful insigfit-s for program, 
planners.. 

Tvaimlng RaaetPed . Training activities were usually skill- 
specific, such as inst ruc'tioa in how to implement a new reading' pro- 
gram* They c.ould occur .pmor to project implementation in the class- 
rooms, during the first year of implementation, and/or after the 
first year. These activities had strong positive -and direct effects- 
on the percentage of project goals achieved ('.12) and on student 

performance (.13) in the areas of both achievement (.11) and behay- 

25 

ior (.12). (See Tables 5.1 and 5.2.) However, training had only 
a small (and not signi f icapit) effect on teacher change (.06) and on. 
continuation. of project materials (.05). Moreover, Table 5.3 shows 
that thu- positive effect of training on continuation of project 
methods losfes its significance when implementation effectiveness 
is controlled for; thus, the effect of training may.be indirect — 
namely, skill-specific training improve^ project implementation, 
which in turn enhances continuation. * Skill-specific training, in 
summary, apparently affects project implementation and student out- 
comes, but does not affect the longer-term p ro j ect!>^outcomes of 
teacher change and continuation. 

These findings can be put . into perspective and interpreted by 
comparing them with other activities that support project training*. 
Tables 5.1 to 5.3 show the effects of each tra^ -?ng support activity: 
classroom assistance, from district resource people or project staff',, 
use of outside consultants, observation . of the project in other 
teachers' classrooms, and staff meetiings. The tables also indicate 
(by bracKets) the combined effects of these activities.' (The' staff 
support activities were combined by taking a. simple sum.)^^ The 
"rationale for examining .their effect in combination is drawn from.^ 
our fieldwork, which S',uggested that particularly effective projects 

^^The variable called ^'Training Received", in Tables 5.1 to 5 . 3 
is the sum-^of the usefulness of training received in three time 
perio.ds (before, during , and after ' the first year of implementation). 
In preliminary analysis, we tried many ways to combine these vari- 
ables; the simple sum proved to be the most fruitful.- 

• ■ ■ ; 12i ■ 
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adopted a training strategy that included all, or most, . of these 
support activities. We will first discuss the findings for the com.-: 
bined support activities and then examine the role of individual 

components. " " 

Staff Support Strategies. Taken together, well-conducted staff 
support strategies had a major positive effect—as did staff train- 
ing — on percentage of project goals achieved (.18) and on student 
performance (.27) (see^ Table 5.1). In contrast to training, however, 
staff support sti^ategles also had strong positive and direct effects 
on the longer-term project outcomes: teacher change (.15) and con- 
tinuation' of project methods (.28) and materials (.22). It seems 
clear that well-conducted support strategies not only reinforce the 
contribution of skill-specific training, but also make, their own 
contribution to fostering teacher change and to promoting sta&f 
'acceptance of project materials. Our field experience suggests, why 
these contributions ara possible and, indeed, necessary to success- 
ful projects . ' ;p . 

First, both training and support activities promote the "task". . 
learning requisite to the implementation .of specific innovative 
techniques, but they aid learning in different ways. Training 
generally focuses on instruction in specific skills the teacher 
needs in applying the innovation; e.g., for projects stressing indi- 
vidualization, teachers .need to master the procedures of criterion 
reference testing or the logistics of diagnostic-prescriptive tech- 
niques. Then, however, they need to adapt these procedures to the 
day-to-day realities of the classroom. Support activities, partic- 
ularly visits from district resource people or- project staff, can . 
provide feedback to make these classroom "corrections." Through 
these support activities, skill-specific training can be "individ- 
ualized" for the teacher in terms of timing and content modifications 
Staff support strategies also can aid teachers in understanding' 
and' applying complex Innovations in ways that standard training — in ■ 
terms of both its content and form- — does not or cannot us'iially dp 
effectively. For example, projects aiming to change the usual pat- 
terns of classroom organization require a high level of mutual 
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adaptation. There are few "recipes" for implementation of differ- 
entiated staffing ot open classroom pro j ects . . Adaptation in this 
c^se typically impli'es.a translation of general principles and atti- 
tudes into classroom practice—a process that can benefit more from 
drawing on outside consultants, from observing other classrooms 
already applying these complex and necessarily abstractly defined 
methods, and from on-line assistance of district and project people', 
than it'can benefit from standard training/. Without such' assistance, 
teachers would be less likely either to implement the innovations 
effectively or to change their own pattern -'and style of teaching-; 
nor would they .be likely to assimilate the methods of the innovation 
in ways that assure its continuation. As one teacher commented: 
"By being part of the project, I have gained invaluable knowledge. 
Because it was a think-plan-dp-revise type of learning^ it was nituch 
more valuable than any graduate class and enabled me to. make real 
changes and improvement . 

Staff support strategies also lend "moral support" to teachers, 
valuable for bolstering their confidence and maintaining their enthu- 
siasm. Teachers need, to feel that they; are not working alone, and 
to be reassured that what^they iare doing is appropriate and legiti- 
mate. We \/ill see later on that other factors also contribute to 
teachers V affective needs, but support strategies— classroom. assis- 
tance from district resource or project staff, the use of outside 
consultants, classroom observation, and project meetings" — can make 
their owv. unique, dual contribution to both teacher morale and to 
the reinforcement of the instrumental effects of other task-related 
factors. - 

In summary, staff support strategies combined with skill- 
specific training appear to have provided tjie feedback, conducive 
climate , and information to.^ support prbject strategies teacher 
learning, and staff morale. Taken together*, staff support strategies 
apparently contributed to the longejQ-rterm pro j ecf outcomes of teacher 
change and continuation. Below we examine their individual contri- 
butions ...... 

Classroom Assistanoe. Table 5.1 suggests that classroom 



assistance from resource personnel had major effects on student change 
(.14), oh implementation (.11) and on cont inuat ion 'of metliods (.09), 
and smaller positive effects on teacher change (.08) and continuation 
of materials (.02). We cap- obtain still other estimates of the par- 
ticular effects of classroom assistance by considering training and 
support activities to be the 0>Lly inputs .determining project outcomes... 
Table 5.5 presents such estimates; it shows the' effects on the depend- 
ent variables (in terms of standardized regression coefficients) of 
va r iabJ .''es mecisuring both the usefulness of training and support 
ar t ivit les. and the amounts of these activities. Table 5.5 further 
breaks down the training variable accprding to whether training 
occurred before clasf^room implementa t ion , during the first year of 
implementation, and/or after the. first year. Comparison of Tables 5.1 
and 5.5 shows that classroom assistance had sirailar effects on most of 
the dependent variables with or without controlling for otfier project 
characteristics or institutional factors. * • 

We earlier suggested the reason that classroom.: assis tance J.s so 
important for the percentage of goals achieved on, £1 project, the 
amount^ of St udent improvement, and the continuation of the project's 
methods. Skill-specific training or workshop sessions usually cannot 
anticipate the form and t'ontent.,of all important staff needs, espe- 

.irially as they relate to particular classroom problems.. , Likewise, 

■ . ■ - <■ 

staff often cannot perceive what they need to know until the need. 

aris-es. For both reasons,' the ataff needs are not always, predictable 
or synchronized wi th " scliedul ed. training sessions. Classroo.m assis- 
tance, particularly by local resource personnel, can make up for 
these iniavitable deficiencies. 

^" It is interesting tha't a number of the coefficients of the 
-jTuount^ of classroom assistance tend to be negative (though not sig- " 
nifirantly) in Table 5.5. At first glance, negatiive Values seem to 
run counter to intuition, but upon reflection they suggest an import- 
ant hypotiiesis about the relationship between the quantity and quality 
of training and staff support. Figure 5.1 is a graph of the percent- 
age of project goals achieved as a function of the perceived useful- 
ness of classroom assistance; the various lines represent levels of 
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V . ■ AMYSIS JF TIUININC AB SUPPORT ACTIVITIES ' , 

1 • ' ' ' ' ' 

. ' ' . Standardized Regresjiion' Coefficient for: 

PercL'iit • 

Project . Total ' Total . , Continuation , Contlnua.tion 
. ; GoaU , Teacher ' Student of Project ' of Project 'Practicalitv 

Independent Variables , Achieved ' Change Improvenent Methods Materials of Training''* 



Whether project had: 



Training before first year, 




-.06 


-.02 


-,06 . 


-.06 


-.12 


Training during first year 


-.00 


'M 


-.07 


. -.05 


-.03 


-* 
-.08 


• Training after first year , 


-.08* 


'M 


-.08 


, -.04' 


'' ,02 


-.08* 


Clas's-roooi assistance' ' 


J it 

-.16 


-.()1 ' ' 


-.01 


•:■ .05' 


, .06 




'Outside consultan,ts 


-.06 


,.03 


-.04 


-.02 . 


-.03 


, -.14* 


Observation of. other classrooms 


-.00 


-.06 . 


-.01 


-.07 


v04 ■ 


.03 


Project meetings 


.01 


-.00 ■ 


-.00 


-.05 


-.00 ■ 


-.04 


(Juantity of: 














Training before Kirst year 


-.02 


.11*- 


-.06 


i 

.09 


''04 


- 07* 


■ Training during first year 




-M 


-.02 , 


.03 ■ 


-.02 


-.05 


Training after first year ' 


,0A , 


.06. , 


■ •.06 , 


.05 


-.08* • ' 


-.01 


Classrooni assistanri* 


-.02, 


■ .06 


' ■ -.02 


-.03 , 


■ -:00 


-.06 


Outside consultants 


.oi' • 


.07 ' 


-.00 


-.02 


-.01 


. -.03 


Observation of other classrooms 


-.07 


.03 


-.03 


-.00, 


-.06 


■ .05 


Project meetings 


-M 


-.10*' 


-.09* ■ 


; -.10* ' 


., , -.07 


.07 


Usefulness of: . 






* 








Training before first year 


.08* 


.11* 


.13* 


.09* ■ 




.04 


Training during first year 


.17* 


.06 


' .10* 


• :09* 


■ .11* 




Training after first year 


-M 


-.02 


.11*' 


-.01 


■ -.02 ■ , 


.16* 


/ ClassrooTD assistaiu'e ; 


.13* 


.10* 


,04 


.10*' 


■ .06', 
-.00 




.Outside consultants 


.23* 


.05 


',13* 


.08* 


.25 


Observation of other classrooms . 


-.03 " 


.03 


,01 


,09* 


.07 . 


■ .01 


. Project meetings ■ .• 




.Oi' 


.05' 


.09 


, ' .07 • 


.24* 




' > 
•35 . 


; .22'* 


. ,32 


., ".27 ■ 


,,,18' 


.61 ' 


Number of observation^ 


498 ' 


498. ■ 


■ 498 


■ 498 


,498 • '■■ 


498 



Significant at the .10 level using a standard two-tailed Vtest. 
^Defined and discussed on p. 112. 
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Negligible 
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High. 
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Usefulness of classroom assistance 



6 7 
Extremely 
useful 



Fig. 5.1 — Effect of classroom assistance on percent project goals 
^ achieved: quality versus quantity of assistance 
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the amount of assistance^ The graph illustrates the significant - 

improvement in percentage of goals achieved resulting, from useful ^ , 

• . * ■ • ' , " , ' / 

assistance". (The improvement is on the order of 50 percent in goals' 

■ ^ ■ i 

achieved, going from an average percent achievement of about 35 pep^ 
cent for assistance that was not usefuV to about 85 percent for^Krery 
useful assistance "However , it also shows that the differences in 
the amount of assistarice do not matter when the assi'starr^e^ is per- 
ceived as helpful or very helpful, but they do matter when the 
assistance is perceived as not helpful. In sharx , numerous visits 
to the classroom by district or project stafF"^ do more harm than 
good when- teachers do not feel they are^b^ing helped. 
4j The inference from these data— a li^ttle high-quality support is 
better than a grea^t deal of low-quality^ support — also applies to the 



other support activities, particularly/ the use of^ outside consultants 
and the frequency of projecD meetings-^ 

Outside Consultants, Table 5.5 sfiiows that the usefulness of 
outside consultants has positive -effects on all the dependent vari- 
ables , whereas the frequency of employing outside consultants has > 
negative effects; This suggests that it is better ' to, use no outside 

consultants than to use poor ones,^and much better than to use poor. 

27 ^ 

ones frequently. When consultants were good, they were very good; 
when they were bad ,. they were an obstacle. 

What makes for a good outside consultant? How do thfey help 
projects? And how do b^d consultants hurt? 

Good-consultants can help by providing concrete , practical . 
advice to project teachers — showing them, either in their classrooms " 
or in "hands-on"' workshops , .how to adapt project methods or materials 
to their own situations. Good consultants help teachers learn how to 



26 

The graph of Fig. 5.1, which was drawn from regressions, 
illustrat;^s the relationship between quantity and quality. This 
relationship can be more fully examined by analysis-o f-variance 
techniques that make no assumption about the cardinality of the 
independent variables. These techniques were used in p^reliminary 
analysis and they confirmed the conclusions drawn in the text. 
' 27 ■ - 

The dummy variables showed that the use of consultants per 
se did not improve the success of the project. 
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solve problems for thomselvesV rather thnn solve problems for them. 
Ineffective consultants often furnisfi adviTe that is too abs tract to 
-he uset iij . Many pro ject teachers commented on tliC'. irrelevance of 
consultants to t^heirc project . For example, in making a recommenda- 
tion for improving project implementation- in the future, one teaclier 
advised, "Be sure consultants know the (project) goals and some 
specific things to tell the teachers and not a .lot of v^orthless 
generalizations and t>heory." Another teacher remarked, "I found 
■ most . [of ■> the consultants) to be completely lacking in Lheir exposure, 
'to, familiarity with, and willingness to come in and work with young. 
children. Many were good philosophically, but riot practically, in 
the day-to-day approach and follow-up." Even vyhen their advice Is 
concrete,, consultants often unintentionally "preempt" staff learning 
opportunities, with too great a show of expertise'. Overreliance on 
outside consultants, then, of ten - prevents teachers from learning 

to do it for themselves." One superintendent attributed- t ne' f a ilufe. 
of a project to t Ids . f ac tor : "The first year, teachers came In from 
other communities and worked wi th- our teachers . The fo-llowing year, 
our teachers weare alone and i,t, was impossible to fully implement the 
program/* Thus, the negative ''Coefficients in Table 5.5 can be seen 
as a result of bo.rh .t.i^o little and too much help from, consultants. 

Project Meetings . ■ Project meetings play* a somewhat different 
and more auxiliary role in' promoting effective projects than. does 
the use of outside consultants. Two of our tables seem to present 
contradictory evidence in'thls regard. According to Table 5.1, 
.meetings are no t.^signif icant and tend^to be negatively related to , 
proiecc outcomes; but Table 5 . 5 "* Ind icat es positive though no^ very 
strong effects of meetings on project outcomes.,. This statisticab 
discrepancy has. two sources. First,, there is the issue of quantity 
versus quality (Table 5.5 controls for quantity and Table 5.1 does 
not): Frequent meetings that .teachers do not perceive as useful are 
associated with less effective projects. (Preliminary analysis of 
variance strongly showed this to be the -case.) Our fieldwork' sug- 
gests that meetings are unproductive' when they dwell primarily on- 
details of, project administration and recordkeeping and rarely. 

. ■ . ; . ■ • ' • ; '" \ ■ ■ ; ■ ' 
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.include opportunities for staff to share their problems and report 
on progress. Such meetings do little to enhance classroom imple- 
mentation, and teachers' f ind them irritating. 

Effective project meetings support teachers in carry In>^ out 
their tasks and meet their affeclive heeds. A*s one . teacher , commented , 
^'Regula r month !.y meet ings*' are absolutely critical far reinforcement ' 
and bui lding interpersonal relationships for co-workers . 
^. , Because project meetings can* be the forum for f?uch critical 
factors as teacher participation in project decisions and project 
director leadership, their u<4efulness is correlated with these 
factors and provides little independent contribution to project out- 
comes. This multicnl linearity is the second reason that Table 5.1 
shbws a slight negative effect of meetings whereas Table' 5.5, which 
does not control for teacher participation and other factors, shows 
a slight positive effect. 

In sum, effective projects tend^ to iise meetings in conjunction 
with other effective practices and attitudes. Such meetings facili- 
tate a' number of activities that are important, to implementation — 
e.g., ongoing assessment of project . activities, staff sharing of. 
problems and solutions, and the building of a "project identity." 

■.Observation of.Mher Classrooms . The statistical difficulties 
clouding the effects of meetings also obscure the effect of classroom 
observation on proj.ec t out comes . Again, Table 5 .J., shows largely 
insignificant coefficients for classroom observation, while Table 5.5 
indicates positive and significant, though not very strong, effects. 
The correlation of classroom observation with other factors — spe- 
cif ically 20 with community involvement projects and .24 with 
classroom organization projects^ — explains much of this- discrepancy. 

Most projects: in our sample did not employ classroom observa- 
tion; for those that did (approximately 35 percent), one-half day. 
or less of observation did not p rove useful , whe reas threb days or 
more proved extremely useful. . ' ' ^ 

Classroom observations can be valuable in giving teachers a 
"feel*' for ^the innovative strategy they are"^ trying tp implement. ^ 
Observing the real thing" for perhaps three or four days is 
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particularly useful to teachers who are about to implement inherently 
amorphous innovations, sucli as change ^in classroom organization (open 
classrooms, team teaching:, differentiated staffing, and the tike). . 
Teachers will learh"> lit tie from too brief an observation, however, 
and may even find the experience intimidating if they watch an expert 
teacher smoothly running an innovative classroom ("I could 'n<5ver do 
that"). - > 

Quality of Tvqini^ig . What constitutes "quality" in training 
and support activities? Some oF the answer is hinted at in the pre- 
ceding discussion. One component clearly suggested by our. field,, 
observation is the "concret.eness" of staff development activities, 
or what we call p?vrat7!c?a7tti/ . Instead of focusing on abstract 
classroom implications of the project, consultants and workshops 
dealt with nuts-and-boJ ts issues, such as what, to do when. Although 
teachers no doubt were interested in the theoret ical underpinnings 
of their projects, they needed timely and concrete assistance with 
their immediate classroom problems. Of the teachers in our sample, 
22 percent mentioned lack of • familiarity with project methods as a 
problem during project implementation — -a result to be expected if 
the. project in fact represented a practice new to the district. 
Similarrly, 21 percent of the teachers mentioned inadequate training 
as a problem; many of them. may have had the same complaint in mind 
as\the former group: the failure of their training to help them, 
gain practical familiarity with project methods. 

We asked teachers 'the extent to which project training and 

support activities focused on practical problems, and found a wide- 

28 

spread distribution of responses. Table 5.5 suggests that 
28 

The specific question and distribution of the 973 responses 
■were: . (Practicality of Training) ^^Considenng all of the ^ ^training y . 
advicGy and assistance you received as part of the project^ how much 
did it focus on the practical problems you dealt with in implement- 
ing the project? ■ 

The training focused on practical problems... 

9% 21% 24% ' 20% .13% 9% 4% 

J ^ 1_ L L_ -1 I ^ L 

7 6 . 5 4 . 3 ' 2 1 

Completely - Not at all 
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practicality is a major component of the usefulness of meetings, 

cLasHroom assistance, and the standard training consisting of, lec- 

tures and workshops.' In other words, the niore practical they are, 

the more useful. ^ . 

Though Important ,. however , concfeteness is not enough. for 

29 

teachers to rate training apd. support activities as "useful"; 
they ipust be timely as well. A workshop on the construction of 
reading materials will not be very useful, if it is held three wetiks 
after teachers have had' to develop the. materials on their own. But 
just as planners c^jnnot accurately, foresee the most useful content . 
of training and support activities, neither can they predetermine 
the best ' sequence and timing of training activities. Again, staff 
meetings can be. the answer if they provide teachers with a timely 
forum for identifying their training and support needs. 

Teacher Participation in Project Decisions . Besides making 
choices about training and staff support activities, teachers may 
be allowed to pard-iciiDate in decisions about projec,t operations. 
Tvpicallv, thev are not involved in., major policy decisions made 
during mobilization and institutionalization by superintendents and 
central of f ice . adminis trators — e.g., whether the project should^be 
adopted, and how project resources should be . allocated . Although 
teacher participation in those early planning actiyi^ies can be 
beneficial, as another section discusses, teachers aro more likely 
to deal wi th . devcisions related to day-to-day implementation problems 

29 ■ ' \- o 

The evidence suggests that practicality iis a component of 

usefulness, not vice versa. Thus, practicality explains much less" 
of the variance in the dependent variables than does the combina- . 
tion of training and support activit.ies. For example, practicality 
alone explains 11 percent and 20 percent, respectively, of the vari- 
ances in continuation of methods and in percent of project goal*s 
achieved compared with 2h percent and 29 percent explained by the 
cpmbined support and training activities. Moreover, the contribu- 
tion that practicality adds to the explanation of these variables 
beyond those made by the training and support ^variables is an 
insignificant (by an F-test at .01 level) .002 and .02, respec- 
tively.^ We can thus infer (a) that practicality is a component of 
usefulness but (b) that usefulness has other , dimensions . To repeat, 
concreteness alone ^is not enough. 
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and, more particularly, witli the adaptation of tlu' innovation to its 
sgttinp. The latter Form of part i i.pat Ion Is tlic subject of the 
following, analysis. * 

l)e('isions and eluvices would not be necessary durin^^ Implemenba- 
t.ion If [)ro jects^ were a I ways , and pred iclably , implemented as ori- ^ 
finally plannt>d. Most of them, liowever, undergo mod i f i c^at ions in 
tiiel r'origi liar s t rategleyv or objectives, and these adaptations are 
\ihfiosL aUv^ays seen as positive improvements. As one teacher observed, 
"Acluai.ly,' a muclr better, program developed to suit our needs than tlie 
one originally written." The projects in (Ujr samj)le varied consider- 
ably In l.hi» exronl-.to which adaptation of the project's methods or 
materials took place, with most t\eachers indicating that their pro- 
ject had been adapted a fair amount . The extent to which teacher?: ^ 
felt they had participated in decisions concc^rning these project 
adaptations varied even more, with as many as 40 .percent of the / 
teachers considering themselves t.o h,'lve been relatively uninvolved. 
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, The ' speci f ic questions^ and distribution of the 975 responses j 

were: (Materials Modification) Tn what extent were project m<xtGrials 

irio:iifi>d, orr ritffcrent material.^ substituted, during the aourne of 

the rvpjccJ? . 

7% 13% 27% 25% ' 11%. 10% 7%' 

.J . \ LJ \ I I 

7 ^6 5 . 4 - 3 2 1 \ 

To a v^ry large ^ Not at ^ 

extent . all . 

(Methods Modification) To what extent loere project, methods., rnodi- 
' yicn duyina the. course of the project? 

4%. - 13% 27% ' 23% 12% 14% 6% 

J— \ I I I I J 

•s. . 7 . 6 . 5 4 3 \ 2 1 

To a very -large Not at 

extent • all 

'31 1 ' 

•The specific question and distribution of the 1030 responses 

weref (Participation in Project Decisions) To what extent did you 

participate in decisions concerning modifiaations in the project?. 

10% 16% . 17% 16% ' ^'^11% 12% : ■ 18% ' 

I I - \ I i< i I 

• <7 6 c 5 4 3 2 a 

To a ;very large Not at 

Extent ■ t . - all" . 
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Tab,li- 5.1 indicates that the more teachers felt they were partici- 
pating, the higher was the percentage of p-iais achieved by the pro- 
ject (.09) and the more extensive was the continuation of both the 
project's meth^ids (.12) and its material s .(. .10) .. . 

The strong positive effect of teacher participation on the per 
centage of the project's goals achieved s»iv,gests that teachers* 
inputs '.:an s ign PKicant ly. improve; implementat ion . One participant 
in a higlvly sucressnrKl^ending project ascribed the project's 
success primarily to such^taff input: "The pro.^ect changed a lot 
over 'the three years, not be^,x3use tht basic c^oals changed * bu^ 
because everyone was encouraged^o.^rrcp.t ribute new ideas and it 
just grew and got better. 'The goal oY^impjoving teaching has > 
aJwnys been kept foremost — just Vhe ways of getfting tti^re Have 
changed . " 

Nonetheless we have often heard school district managers 
advocate staff part icipat ion beca.ise it satisir'les the staff's "emo- 
•tional" — tir political — need not to be ''dictated to." Too often, 
the .resulting participatory devices are mere window-dressing that ^ 
few teachers take seriously, 

, Our data and field observations indicate that real teacher 
participation in implementation decisions can help solve many prac- 
tical problems. For example, a number of district officials have 
commented that teachers are often more astute than ^-.entral office 
specialists in selecting material appropriate for staff arid stu- 
dents, and In pinpointing why a part icular educat ional . s t rategy is"" 
not fulfil ling expectations. Likewise, project directors and 
principals have often noted that ^project teachers ,are in the best 
position . to perceive their training needs and identify the most 
effective way to meet them. One elementary school principal 
advised, "Hive the classroom teacher a strong role in planning 
any project' that :he or she is going to be working with. Then 
listen and change when things do .not go as planned on paper 

The hypothesis that teacher participation has instrumental 
value — i,e., it improves the implemented project—does not deny the 
affective value that participation has. for teachers. In our sample, 
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•• teacher participation usually involved frequent formal and info rmnr .- 

interactions among the project staff and, as expected, was associated 

'32 

with good working relationships among teachers. Its positive effect 
, on continuation, however, resulted from another of- its salutary bene- 
^fits: the development of teachers' "sense, of ownership . " 

One field observation strongly underscores the importance of a 
sense of ownership. When teachers come to regard a project as "theirs 
(as opposed to !'the project director's" or "the superintendent's")" 
they work hard to implement it, integrate its strategies or materials 
into their classroom practices (often in areas not directly related 
to the project), and take pride in their accomplishments. They become 
excellent emissaries for the project and for innovative efforts gener- 
ally. Such teachers are very likely to keep on using project methods 
or materials whether the project is fomally continued or not. 

The sense of ownership was an important by-product of local mate- 
rials development activities, wherein te.achers developed their own 
materials either from scratch or by redesigning or resyntheslzing com- 
mercially available materials. Projects that included local develop- 
ment of materials (77 percent) achieved somewhat higher sc6res on 
student performance and teacher change than .projects which did not, 

but the most important result was that the teachers were more' likely 

33 

to continue both the materials and the methods of the innovation. 
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A great deal of social-psychological literature on small group 
behavior points to the interrelationships among participation, inter-, 
action, and good working relationships* The next section explores 
the effects of good wxDrking. relationships . Teacher participation and 
the frequency of project staff interactions was correlated .40 for 
our sample. The question dealing with the frequency of project staff 
interactions and the distribution of the 1036 responses were: (Fre- 
quency of Staff Interactions) How much did you consult with project 
teachers on an informal basis? 

] 33% 23% ^ 16% 13% 8% 5% 3% 

J ! I I I I _L 

7 6 5 4 3 2 1 

i^A great Not at 

deal all 
33 

This assertion is an inference from two pieces of evidence: 
first, the uncontrolled (bivariate) relationship between whether a 
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In summary, teacher participation appears to benefit implementa- 
tion and to promote continuation of pro;;ect methods^- and materials in 
at least two iijiportant ways. First, because of their close day-to- 
day involvement with project operations, teachers are in a unique 
position to provide feedback about project activities and suggest 
effective corrections or modifications. These "reality-based" modi- 
fications appear to promote effective project implementation and thus 
the percent of goals achieved. Second, participation appears to 
endow teachers with a sense of ownership that inspires them to work 
hard for project success and to continue using project methods or 
materials. " ' ' < ^ 

The Strategy of Mutual Adaptation . Thus far, we have reviewed 
the various effects of the individual implementation strategies 
chosen bv local planners. These strategies are not chosen without 
reference to each other; they combine to make up the project^s over- . 
all implementation strategy. Although the "mix" of implementation 
activities could and should differ from project to project, our data 
also suggest that certain features are requisite to any effective 
implementation strategy. We examine them below. 

An implementation strategy that focuses only on skill-specific 
training is not adequate by itself to promote the longer-t^rm out- 
comes of project continuation and teacher change. True, such training 

project did or did not develop its materials and this' extent of pro- 
ject continuation and methods (.16) and materials (.13); secondly, 
the significant, though weak, value of the partial correlation of 
local materials development with the continuation of project methods 
after the strong effects of participation, frequency of staff inter- 
actions, and the extens iveness of modifications to the project are- 
controlled: . * 

Partial Correlation 
with Continuation Signif. 
Independent Variable of Methods , at 

.01 .83 
.18 .00 
.19 .00 

.07 .04 



Extensiveness of modifications 
Participation in project decisions 
Frequency of staff interactions 
Project had local material'^ 
development 
2 

R = .35 / 
. N = 563 
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is necessary for task learning during . impleroentat ion , and is positively 
related to change in student performance and thu percent of project 
goals achieved. Our data suggest, however, that these gains may prove 
transil;ory unless training -is combined with other s''taff support and 
participatory strategies, particularly teacher participation in deci- ., 
sionmaking and local materials development, timeJy staff meetings, 
and classroom assistance from resource personnel. These components 
can provide the climate and fieedback necessary to mutu.il adaptation. 
In sum, local planners should take a flexible approach that provides 
the information necessary for implementation, and the project support 
arid staff involvement necessary for adapting and . assimi lating that 
information. 

We have now explored three , of the change agent project's char- 
acteristics; educational method, scope of proposed change, and 
. implementation strategies. To interpret the effects of these char- 
acteristics, the discussion made reference to the schooi district 
context of the projects. We turn next to this context, the insti- 
tutional setting.,. 

INSTITUTIONAL . SETTING 

Tables 5.1 to 5.3 confirm a view held by practitioners to be a 
commonplace truth: The-instltutional setting— -the quality of the 
staff, the leadership of administrators and project directors, and 
. the organizational climate ufschools — powerfully affects project 
implementation, outcome^ and continuation. We • do not wish to 
belabor the obvious here, although.lt may be appropriate to do so 
considering how frequently, federal and state programs neglect the 
importance of the institutional setting. Instead, we wish to differ- 
entiate among the effects of the various components . of the organiza- 
tional setting, so that policy can be based on a more precise notion 
of what can and canndt be changed, and how features of the' ins t i tu- " 
tional setting can be expected to influence the results of federal 
change agent policies. . . « 
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• Organizational Climate 

Phase T founcf-'that organizational climate can impose subtle yet 

powerful constraints on implementation. For example^ if project 

teachers sensed that their principal was indifferent to project 

goals and their own efforts, even a skilled project director might 

be unable to rouse them to make the extra effort needed for project 

success. The results were likely to be lackluster at best, whereas 

the same project could score impressive successes at a site where 

leadership was. dynamic and morale was high. 

Accordingly, Phase II made a closer scrutiny of three vital 

facets of climate: the working relationships ^among project "bleachers , 

the role of* the principal, and the effectiveness of the project 

director. The following discussion explores the effects of these 

factors, their interrelat ionships , and the. extent to which they are 

"immutable" or amenable to change. 

Quality of Working Relationships Among Teachers . Working 

- • ' 34 

relationships significantly affected project outcomes.. Table 5.1 

shows that tHe better they were,, the higher was the piercentage of 
project goals achieved (.22). They were also positively related to 
the continuation of project. methods (.13) and materials (.15). 

Our fieldwork suggests two related reasons for these findings. 
For Qne , close coordination among teachers is necessary in projects 
featuring curriculum revision, team teaching, or differentiated 
staffing. Without it, the cooperative planning and staff inter- 
dependence that are fundamental to such innovations could not occur. 
Second, even if an innovation does not require teachers towork 
together, the development of a critical mass of teachers allows them 

34 ~ 

The question asked of teachers and the distribution of their 
1030 responses were; (Quality of Working Relationships) \How would 
you desci*ibe the working relationship among teachers on this pro- . 
Ject? . ' ' • ^. . . ■ 

36Z 32% 19% 11% 3% 



Excellent 'f. Very Good Fair Poor* 

. Good 
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to openly share their implementation problems and solutions, and. thus 

learn from each other. .Good working relationships- also enhance 

morale. Moral support can be invaluable for " teachers , part icularly 

35 ■ . ' 

in the difficult and exhausting first year of a project. 

How do good working relationships develop on a project? To 
some extent they result, from teacher, participation in project decij- 
sions and the frequent' interaction of staff. (Table 5.6 shows that 
the correlations between the q • ity of working relationships and 
these factors are .33 and .50, respectively.') The data also suggest 

that they depend on organizational cl imate, in regard to which the 

' ' 36" 
survey asked ti?achers a series of three questions. In addition. 



35 

Good working relationships among the project staff are nega- 
tively correlated with the difficulty of .implementation — that. is, 
the better^ the working relationships, the less difficult was imple- 
mentation. (Inde.ed, it has the highest negative correlation^ of all 
the factors used in Table 5 . 1 , whe ther one considers bivatiate rela- 
tions hip.s. or partial correlations.) This correlation can be inter- 
preted in two ways: (1) good working relationships enabled f.achers' 
to overcome difficulties, or (2) projects difficult to implement 
could not develop good working relationships* Both of these alter- 
natives undoubtedly occurred in our sample. 
36 

The survey asked: Please indicate whether. you agree or dis- 
aaree with each of the following statements... 

N*either *^ ■ 

Strongly Agree nor Dis- Strongly No, of 

Agree Agree Disagree agree Disagree Responses 

(School Good Place to Work) 
I»; general,' this 

school is regarded ■ ' 

as a good place for 
teachers to work 
compared with other 
schools in this . , 

district . v,^... A2% V 36% 1.6% . 6% 1% 982 

(School has ^Group Spirit) ^ - . * 

There^s not much • '[\, 

group spirit in 4 ' 

this school 4 . 11 15 * 37 . '33 -978 

.(School' was Task-Oriented) . 
This iti an effi- 
cient, work- 
oriented school.. 29 . 46 16 6 3 980 
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Table 5.6 i 

CORRELATIONS OF SCHOOL CLIMATE AND PROJECT RELATIONSHIPS 

. . ^ Variable Number 



Number 


Variable Name 


.(1) . 


(2) 


(3) 


(A) 


(5) 


(6) 


(7) 


(8) 


(9) 


(1) 


School good place work 


1.00 


















(2) 


t'chool had group spirit 


.AO 


















(3) 


S-ihool was task-oriented 


.39 


.A5 
















(^) 


Effectiveness of principal 


.AA 


.AO 


:37 














(5) 


Principal support 


.28 


.28 


.30 


.Al 












.(6) 


Quality of working relations 


.18 


.29 


.32 


.15' 


.A3 










(7) 


Frequency of staff Interactions 


.08 


.18 


.22 ' 


.OA 


.19 


.50 








(8) 


Partlc ipat ion in project dec is ions 


■ .12 


.09 


.05 


.11 


.15 


.33 


.AO 






(9) 


Teacher encouraged to' modify project 


.08 


.11 


.13 


,.11 


.17 


.28 


.37 


.60 




{10) 


Methods modification . 


.06 


.10 


.05 


.10 


.10 


.10 


.23 


.AA 


.51 


(11) 


Materials mod If icat.ioti . . 


^o7 


.00 


.06 


-.03 


.13 


.27 


.30 


.A6 


.36 



.58 



NOTE: Number of observations is 826. 



because Phase I documented the importance .of the principal to the 

climate of the school, we asked teachers to evaluate how effective a 

37 ' 
job their principal was doing. Table 5.6 shows that. these measures 

ar.e highly intercorrelated as we expected, knd also that they are all 
.related to the teachers' feeling about the q^uality of the project *s • 
working relationships. ' These data suggest that change agent pro- 
jects, even when they involve only a. portion of teachers at a school, 
are influenced 'by the climate and leadership of the. host school. 

Table 5.6 also displ ays the .correlations among the quality of . 
project relationships and various measures*-of teacher par ticipatioh . 
The associations among these variables can .be made explicit by using 
the statistical, procedure of factor analysis-, which groups the various 
measures, according to their intercorrelations , into "underlying dimen- 

. .. ' 

sions. Table 5.. 7 presents the results of the factor analysis; it 
indicates two significant factors which because of their "loadings'* 



37 ' 

The specific question and the distribution of the 986 responses 
w^re: (Effectiveness of Principal) Overall^ how, effective a job would 
you say your pnncipal is doing at this school? . ■ 

^28% 31% 16% 17% 3% . 3% .2% 

i I I L_ I I ' I 

■ 7 6 5 4 3 2 1 

Outstanding Average Below 



ave rage 
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Table 5.7 

FACTOR ANALYSIS OF SCHOOL CLIMATE AND PROJECT RELATIONSHIPS 



Variable. Name 



SchOQl/good place to work 
School had group spiric 
; School was task oriencied 
Effeccivenes s ,o f principal 
Prln'olpal suppipvt 
Qiralicy of working rel at ions 
Frequency of staff in cerac c Ions * 
I^ar c ic ipac ion in project decisions 
Teacher eni'Duraged to modify projiM't 
Methods modification ' 
Materials .mod if Itrat ion 



-Sum of squares 



Rotated Factor Loadings 



' Projec t * 
Adaptat ion 



Interaction 
School of School 
Morale and Project 



.075 

.061 

.008 

.020 

.023 

.117. 

.280 



.744 
.699 
.678 
>771 



.744 
.678 
.857 
,778 



.484 

.''2Y7"^ 

.012 

.049 

.097 

.038 

.009 



.028 
.152 
.170 
.009 
."4'24 



rm 

.763 



.312 
.293 
.059 
.066 



2.454 



2.354 



1.678 



Commonality 



.560 
.474 
.489 
.595 
-.415 
.736 
.661 
. 654 ' 
.555 
. 739 
.609 



6.486 



^Loadings ref^r to- the correlation between an edur.'it ional teclinique and a factor of 

general educational method. Tlio .boxeu entries indicate high loadings on a factor that 

serve to define the meaning of tlie factor. Commonality and sum of squares ar<^, statis- 
tical measure^^j^ Indicate the technical validity of the analysis. 

could be called project adciptation and sohool morale^ and a third 
significant factor that apparently reflects the interrelationships" 
between the project and the school. The quality of project working 
relationships is the main variable defining the third factor. (The 
variance of project working relationships is also explained by the 
first (.12) and second f actors (. 23) , with the largest weight coining 
from the second factor — i.e., the school climate.) In short, good 

project working relationships come from good schools and from adap- 

38 . 
tive projects. ■ " . - • 



More generally, we suspect that a projecf's working relation- 
ships are a function of organizational climate , implementation , and 
the interaction between the two. Therefore, project outcome's are 
both effect and cause of the project's working relationships, and, 
.consequently, the estimates of the-effects of the quality of working 
relationships oh project outcomes are subject to a' positive bias. 
The same type of bias is present for the other organizational cli- 
mate variable, principal support and project director effectiveness. 
We could have eliminated this problem by not using these variables, 
but. that seemed too costly for exploratory . purposes . Instead we 
retained these variables and examined the sources of their variance 
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In' summary, .our data indica.te that good working relationships 

among teachers enhance implementation and promote continuation of 

39 ■ ' " -. 

project methods and m.aterials. Good working relationships and 

teacher participation in project decisions are correlated: the 
development of the one helps the .development of the other. And, in 
addition, the quality of the school's organizational climate — whe.ther 
teachers feel Jtheir school is a good school to work in, has esprit de 
corps, is efficient, and is managed effectively -by the principal — 
influences the quality of project relationships.. The correlation* 
"between participation in project decisions and good staff working ' 
relHt ionships draws attention to the implementation strategies chosfen 
for the .project , the influence of the general school climate — a back- 
ground factjor not dir'.^ctly related to project opera tions^-underlines 
the significance of district. site selection. Good project working 
relationships can develop in "average" schools when teachers parti- 
cipate in project adaptation decisions; and, conversely, "good" 
schools can develop good project working relationships v/ithout 
teacher participation in . decisions .■ However, projects combining 
a supportive organizational environment with a strategy of teacher 
participation in project adaptation seem most able to implement 
effectively and continue their innovations. 

Role of che> Principal . Our previous reports referred to the 
principal as the "gatekeeper of change," in recognition of his or her 
vital role in either facilitating or inhibiting innovation. Phase II 
data corroborate that finding, and further suggest that project con- 
tinuation depends critically on the principal's support. 

other than that explained by project implementation. See App.B for 
an {inalysi.s of covariance of many of these variables that examines 
how much of the variation • can be explained in terms of the teacher's 
.school or project- 
39 

A number of teachers viewed the promotion of good working 
relationships as one of the most important benefits of the project. 
One teacher said, "The project did so much to get teachers working 
tiogether that I believe that accomplishment alone would have justi- 
fied the . proj ec t . " Another comment ed , "The most important thing for 
rne is that this project has worked to bring our faculty closer 
together in terms of philosophy, concepts- and our everyday dealings, 
with each other." 



Th^ Phase II survey asked .teachers to indicate the attitude o£ 
their principal toward the project/^ Few of the projects in which \ 
principals were perceived to be* unfavorably inclined toward the inno- 
vation scored high on any of our outcome measures (see Table :5.1). 
Some projects with neutral or indifferent principals scored well, 
particularly in the percentage of goals achieved; but these projects 
typically focused on individualization or curriculum revision, and ' 
had highly effective project directors who .comj^ensated for the luke- 
warm principals. Projects having the active support. of the principal 
were the most likely to fare well/"^ - As. Table 5.1 shows, the more 
supportive the principal was perceived to be, the higher was the 
percentage of project goals achieved (.11), the greater the improve- 
ment in student performance ( . 13) , , and. the more extensive was con- 
tinuation of project methods (.12) and materials (.09>. 

Practitioners and policymakers . are well aware that , principals 
are critical to the quality of school life as. well as to project 
outcomes. But these data allow us to go beyohd confirming this 
general truth to explore more detailed questions about the different 
(and more or less effective), roles played by principals in implemen- 
tation and continuation. 
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The specific question and distribution of the 993 responses 
were: (Principal Support) How would you describe your current prin- 
cipal's attitude toward the project?. • ^ 

• 37% 19% 15% 8% 16% - 1% 1% 1% ,1% 

J ^ I L i i I I I I ' 

. 9 . .8 .7 6 5 4 3 2 .1 . 

Extremely Neutral or Very 

favorable indifferent unfavorable 

4i„ ^ ^ - - . 

We refer to. the measure of the principal's attitude as the 
"principal's support'Vof the project becausi. it is highly correlated 
with the teachers^ responses to. pur questions about the support given 
by the principal, and how helpful the principal was. For example, the 
correlation between the question referred to in the preceding foot- 
note and a question asking about the "helpfulness" of the principal 
was .79. As. we suggested in an earlier footnote, because principal 
support may be both cause and effect of project outcomes, the coeffi- 
cients in Tables 5.1 to 5.3 may be positively biased. 
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Though principals . In our, sample were as supportive ot favoratily 
inclined toward one type of innovation as to any other, their support 
for a project— and their actions in it — mattered dif f erent ly " for 
different kinds of educational treatments. For example, we explored 
the difference that a principal's^ support might have for classroom 
organization projects as compared with projects employing individu- 
alization techniques. Table 5.8 shows the result^ Of regr:essing our 
project outcome measures oh most of the relevant factors used in 

Table 5.1 for two sobsamples: pro j ec ts focusing on individualization 

42 ' 

and projects focusing on classroom organization. ., The row report- 
ing the partial- correlations for ^'Principal Support" ^indicates that 
(a) supportive principals significantly increased the prospects for 
continuation of both types of projects, and (b) supportive principals 
significantly helped the proj ec t ' s ' implementation and increased the 
extent of improvement in student performance for classroom organiza- 
tion projects, but had a smaller (and riot significant) effect on the 
implementation of projects focusing on individual ization . 

Let us first discuss why the principal ' s " support is so, important 
to continuation, independent of what kind o^f innovation may be 
involved. At . the. end of federal funding, the . principal must take 
a stance toward the project and make a variety of decisions that 
explicitly or implicitly influence what happens to. project methods 
and materials withiri the- school. ' In part icular ,' the principal is 
cliiefly responsible for establishing the school's educational poli- 
cies and philosophy. A project that is consonant with the school's 
general operating style would be more likely to be sustained or 
spread than one that was not. . For example, we observed an open 
classrobrr. that operated in a very traditional school as !part of a 
districtwide open-classroom project. Once the umbrella of project. 

~' 42 ^. 

The subsamples were created using the factor scores for Indi- 
vidual izat ion \Tiechniques and Classroom- Organization Change.' One sub- 
sample consistejd of all projects with a score of at least 1.2 stand- 
ard deviations on the Individualization Techniques factor; the other 
sdmple consisted of all projects with a score of at least .9 standard 
deviations on the "Classroom Organization Change factor. A total of 
9 projects fell into both subsamples. 
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Table 5.8 



COMPARATIVE INFLUENCE OF PRINCIPAL AND PROJECT DIRECTOR ON 
INDIVIDUALIZATION AND CLASSROOM ORGANIZATION PROJECTS 



♦ ■ 




Dependent Variable 






Percent Project Goals Achieved 


Continuation of Project' Methods 






Classroom 




Classroom 




Individualization 


Organization ' 


Individualization 


Organization 


Independent Variable 


Subsample 


Subsample 


Subsample 


Subsample 


Principal support 


.16 


■ 

.39*** 


.33*** . 


.34*** . 


Project director effectiveness 


.28*** 


j-3 


-.09 


-.32*** 

f 


Funding level of project 


. •10 ■ 


-.18 


■ .16 , 


I 

' -.13 


Overall- change required 


■ .10** 


".10 


-iUj 


.13 


Extra effort required . . 


-.07- 


-.04 , 


.10 


.36*** . 


Specificity of goals .. 


-.07 


■ .08 


,.05 


/ n AAA 

M 


Number of project schools ; 


-.05 


' .05 ■ 


03 i 




Staff paid for ^training 


-.23** 


.00 


-.09 


.04 , • 


iidining recciveu 


.21**, 


.19* 


-.17* ■ • 


• .20* ■■, 


Participation in project decisions 


.02 ■' 


-.06 ^ 


.15 


.15 


Quality of working relations 


.32*** 


.19* ' . 


.04 :, 


.16 


Elementary school 


■ .15 ^ ' 


.21* 


-.03 


-.04 


.School size 


.12 


-.05 


-.09 


-.09 


School socioeconbmic setting 


■.07 


-.05 


.03 ■ • 


-.,12 ' 


Efficacy (teacher) 


.09 


.14 


.09 . 


: .04 ' 


Experience (teacher), 


-.00 


■ .04' 


-.00 


' -.08. 


Verbal ability .(teacher) 


. -.09 


■ .09; 


-.02 • 


.04 


2" 2 
R /corrected R 


.56/. 48 


.48/. 38 ■ 


.43/ .33 " . ■ 


.57/. 48 


Number of, observations 


116 


99 ^ 




.99 . 



NOTE: Significance for a standard two-tailed t-test: 
* = .10 level; ** = ..05 level; *** = .01 level. 



authority was removed, the principal made it clear to the project ' 
teacher that he wished to see her -classroom returned to th^ tradi- 
tional pattern; he also, strongly discouraged nohproject teachers who" 
expressed interest in trying some of the open education ideas in 
their classrooms;^ In the same proj ec t , however, the- principal at 
another school strongly supported the open-classroom approach. After 
the project ended, she encouraged its use in other classrooms by 
allocating discretionary money to/'knock down walls" and to purchase 
the manipulative materials required^ In shorty, the principal gives ■ 
sometimes subtle but nonetheless strong messages concerning the 

r ' . . • 

"legitimacy" -of continuing project' operations in the school—a 
message that teachefs cannot he]q) but receive and interpret in terms 
of their professional self-interest. Although project methods or 
materials can be. continued to some extent "behind the classroom 
door," It would be extremely difficult for a teacher to continue 
them fully without the principal's explicit support. 

Principal support for' a project is important because of the 
staff turnover experienced by most schools. If proj.ect strategies 
are not to wither awa:y over time^^ the principal will have to famil- 
iarize. new teachers with project concepts and .techniques'. One super- 
intendent observed, "A large turnover in staff [makes it .hard to] 
sustain volunteer activities.' If you get a principal who isn't in 
agreement, with project philosophy, it can be difficult to keep a. 
I^rogram in a school." Conversely, the enthusiasm of principals can 
be an "important element in introducing the proj.ecj to* new sites . 
A superintendent commented, "This project has really been sustained 
through the discussion and enthusiasm of principals. They were tre- 
mendously enthused, at first particularly, so the project spread to 
other -schools . " , 

.Whereas the principal plays a major and unique role for project^ 
continuation that is essentially similar for all types of educational 
treatme^nts, the principal ' s' functions in implementation differ across 
types of innovations. For example , individualization techniq.ues 
require teachers to master specif-ic procedures for diagnosi?? and 
prescription. Typically, these techniques are extensions of existing 



classroom practices, not fundamental changes that indirectly or dir- 
ectly affect school operations in general. In this instance, pro- 
ject directors are best able to supply the detailed, practical "how 
to do it-' advice teachers need to adapt the techniques to their 
classrooms. Furthermore, the /principals could confine their efforts 
to explaining the new techniques to parents and teachers not involved 
in the project, and to lending encouragement- to project participants. 

For two reasons, however, the principal miist be much more active 
in che implementation of complex and comprehensive innovations. For 
one, the principal's cooperation and administrative skill are often- 
indispensable to projects such as those involving innovative changes 
in classroom oiiganization , which entail the accommodation of pre- 
vious procedures to new routines and practices, suchi as nongraded 
systems, the use of parent or senior-citizen volupteers, new staffing 
.arrangements, and irregular class schedules. Second ,\ comprehens ive 
innovations such as open education usually embody a particular 'edu- 
cational philosophy (child-centered education, for example). Unlike 
individualization techniques , which can usually fit into a broad 
spectrum of educational approaches, these classroom organization 
projects. can be viewed ^as a radical and undesirable departure from 
the school norm unless the . principal actively supports them and runs 
interf^refnce with disapproving nonproject teachers or parents. Impor- 
tant' as the project director is, it is unlikely that classroom organ- 
ization projects could be implemented without the principal's active 
support . 

One indication of principals' active involvement is their 

• ' • ' . '■ 43 • 

attendance at teacher training sessions. -. Their particijjation 

'' 43"" 

The specific question asked of principals and the distribu- 
tion of the 160 responses were: (Principals Attend ..Training) What 
pvopovtion of the teacher* training sessions for this project^ if- 
any^ did you attend? 

27% 21% 23% 13% 14% . ' 2% • 

L _^ J i I -u'S^""' 

All Most Some A few * None * naa no 

training 

session 
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updates their classroom skills and knowledge, and thereby better 
equips them to lend adviqe and a sympathetic ear to teachers. At 
a less explicit level their attendance imparts some important mes- 
sages to teachers—notably, their personal commitment and their view 
of the project as a team effort in which everyone is .expected to 
cooperate and work hard. Their attendance therefore helps to under- 
mine the "deficit" model that sometimes colors staff training activ- 
ities and builds resentment ; moreover, it bolsters the notion that 
participation in the innovative project will foster professional 
growth. 

Principals' actions in a project reflect their general style of 
school management . We asked teachers to evaluate the overall effec- 
tiveness of their principals and received a wide range of responses; 
28 percent^ rated their principals as "outstanding," and approximately 
25 percent evaluated them as average to below average, (see footnote 
37). In reaching these judgments, teachers were clearly assessing 
principals along a number of dimensions. To explore the basis for 
the teachers* evaluations, we also asked them to characterize their 
principals in terms of four management styles that are often men- 
tioned in the organizational literature as important leadership 
attributes: the principal as peer, as a provider of moral support , 
as an instructional leader , and as an administrator (see Tatle 5.9 
for the specific questions) . 

What mix of roles or leadership styles characterizes an "effec- 
tive" principal? Table 5.10 displays the "weight" of each management 
style in the teachers evaluation of their principals * effectiveness. 
The data, which must be treated as only suggestive because of the high 
intercorrelations among the responses j indicate the following:. 

; 1. A total of 83 percent of the teachers said their principals 
generally treated them as equals. Principals who were judged not to 
relate to their staff in this manner were somewhat more likely t6 be 

Elementary school principals were evaluated significantly 
higher on all of the dimensions than were junior or senior high 
school principals. 
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Table 5.9 
TEACHERS' EVALUATIONS OF PRINCIPALS 

Educational veGearohers and practitioners have often ootmented on the key 
jK)lc played by the principal in the life of .the school. In the folloidng 
set of queotionSj please indicate whether you agree or disagree with each 
of the statcmentG about yoia* school's principal, . . 

No ither 

Strongly , Agree Nor. Strongly 

Agree Agree Disagree Disagree Disagree 

(Peer) • ' ' • ■ 

^0% ' m 8% It 2% ' 985 

38 ' ^2 .13 6 I . 980 

18 ' 31 32 . ^'^ ' ^ 985 



9 24 26 30 "11 992 

evaluated as "below average" in ef f ec t i veness , But, generally speak- 
ing, variation in this particular management style did not signifi- 
cantly affect teacher assessment of principal effectiveness, control- 
ling for other leadership attributes.. 

2. Although teacher perception of an egalitarian climate within 
the school did not appear to be necessary to effective school , manage- 
ment , the extent to which teachers believed the principal was inter- 
ested in them' as persons contributed significantly to their evalua- 
tion of principal effectiveness. The data clearly show that the more 
teachers felt . that the principal was inter.ested in them and their, 
problems, the higher they rated their principal. 

3. The other management style strongly related to perceived 
effectiveness was the principal as an instructional leader. Although 
somewhat less than half of our sample of teachers saw their principal . 
as functioning in this role, the more the principal was- perceived as 

a source of new ideas, materials, ^nd methods, the higher the 

' * * ■ 

- T5P, 



No. of 
Responses' 



In general, , our principal 
treats me as an equal 

(r ral Supporter) 
Our principal is interested 
in me and my problems 

I Ins true tio>iaL .Li-;ader) 
I think of our principal as 
a source of new materials, 
Idea.s. -and method 

(Adm in ii:tratur) 
Our principal spt-nds most of 
the day handling administrq- 
t ivf* concerns and discipline 
rather than dealing with 
issues of curriculum and 
classroum teaching methods . . 
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Table 5.10. 

TEACHERS' EVALUATION OF PRINCIPALS' EFFECTIVENESS • 
REGRESSED ON PRINCIPALS' . MANAGEMENT STYLE' 

Dependent Variable:.'," ' 
Prlriclpal Effectiveness 




" Regressloriv^,.;S.t5udard Bivariate . 

Independent Variable Coefficient Error Correlation 

Principal's management style • 
(according to teachers) was: . * 

Peer .09^* .064 ' .56 



Moral supporter '^^a .072 .69 

Instructional leader '^^^ .057 , .70 

Administrator -.11 .OAA -A9 



NOTE: r2. = .59; N = 448. 

* = significant at the .01 level. 



principal wa.i evaluated as an effective school manager. 

4. About 1/3 of the teachers thought their principal functioned 
primarily as an administrator, spending most of the day handling 
administrative concerns and discipline. Principals who were per- 
ceived to operate in this role tended to be rated a^s average or 
below •average in overall effectiveness. 

In summary these data provide insights concerning the complex 
and difficult role of an-effective principal in today's school setting. 
The traditional notion of the principal as simply the building admin- 
istrator who "runs a tight ship" appears inadequate; so. does the con- 
cept of a nonauthori tarian but essentially passive manager. These 
data suggest that the effective principal-combines an active role 'in 
shaping the school's instructional program with a personal and sup-, 
portive relationship to the staff. ^ Taken together with the evidence 
concerning the principal's role in innovative projects, these<data 
also suggest ways in which participation of principals in change 
agent project activities can enhance their overall effectiveness as 
school fnanagers. Specifically, their participation in project train- 
ing sessions can enhance their pedagogic skills and build mutually 
rewarding interpersonal relations with the staff. ,^ 
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The Project' Director . Our data leave little doubt that aa 
•.effective project director, greatly enhances the implementation of, 
special projects. But in sharp contrast to the findings about, 
principals , project directors had no significant eipfect on continu- 
ation for our sample of projects. . . 

We asked teachers to evaluate the effectiveness of their pro- 
45 • 

ject director. The distribution of their answers ranged widely;, 
•about 1/3 said their project director had been extremely effectiveV- 
while aibout 1/4 rated the project director as average or less. 
Table 5.1 shows that, controlling for the other factors influencing 
the innovation, the more effective the project director, the higher 
was the piercentage of project goals achieved (the coefficient is .25) 
and tKe greater was student improvement (.12); the effectiveness of 

project directors made little difference for continuation of methods 

■ f ■ 

(.03) or materials (.01). 

■ \ - ■ ■ 

. Why does the project director "matter" for project implementa- 
tion but not for continuation? The answer can best be understood in 
light t)f the different tasks and activities in the implementation, 
and. continuation phases. K central aspect of implementation is 
teachers' acq.uiring new skills, behavior., and attitudes — task- 
specific learning that an effective project . director can greatly 
facilitate. The director's special skills and knowledge can clarify 
project goals ;and operations, minimize the day-to-day difficulties, 
"encountered by classroom teachers, and furnish the concrete informa- 
tion they need to learn. The high .correlation between "specificity 
of goals" and project director effectiveness was .41. The high 
correlation between useful assistance in the classroom and project 
director effectiveness was .44. r ..^ 



45 

The specific question and distribution of the 980 rjesponses 
were: (Project Director Effectiveness) How effective has the project 
director been in helping to implement this project? 

34% 21% 16% 12% 5% 6% 4% 



Extremely Not at all 

effective effective 
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Once federal funding ends and continuation begins, however, 
tfjsk-learning is no longer a. major staff, activity . . (If teachers 
have not learned project strategies and methods by then, it is 
unlikely they ever will . ) .The activities central to continuation 
comprise integration of project precepts into routine, classroom 
activities and, in some cases, the modification of standard insti- 
tutional procedures. Thus, in this phase, the specialized skills 
and knowledge of'the project director became less, important to pro- 
ject teachers than t-he institutional support of "the " principal and 
other district staff. In fact, it is" not unusual for project 
directors to assiMne an .entirely new role after the end of federal / 
funding. Many. either resume their former position in the district, ;^ 
go on to head innovative projects in other districts, or, in the j 
instance of Title III "validated*' projects, launch upon heavy j 
schedule of project dissemination. * 7 

Thus, theproject director affects continuation indirectly, / 
through the Lniplemented project. UTien. the project director has l/een 
effectjuve, the task-learning central to implcnientation is complete 
as the project enters its continuation phase. Otherwise ,* pro jec t 
activities are likelv to falter badlv or break down once federa'^l 
support ends . / 

^Structural Ch iracterist ics of Schools 

Because our sample was not representative of all th;.- various 
kinds of school settings, v^e gathered data on a wide variety of 
school characteristics and controlled for them In the analysis. Tror 
example, we collected information on the academic, ethnic, economic, 
an4 soci al . makeup of 'the school's s tudent population, the size of 
the school and stability of the staff^ and the school's experience 
with other innovations. (Similar data were gathered at the district 
level.) . None of these ^'background" or structural characteristics 
strongly affectec] any of the project effect or continuation measures. 
Our noti-rrepresentat ive sample precludes our drawing generalizations 
about this lack of significance- We suspect, however, that th^se 
backgroiAnd characteristics matter less for project outcomes than do 
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other factors—the attributes of the staff, the leadership of the 
principal, the effectiveness of the project director, and the organi- 
zational climate of the school. 

Perhaps the most important structural charactr.rist ic with respect 
to introducing and suscnining change is whether the school is at. the , 
elementary or secondary level. Our original Phase I sample contained 
projects at 81 junior or senior high schools; we eliminated 61 of 

them from the Phase II sample because they were totally discontin- 
46 

ued. The remaining secondary school projects are unusual in that 
they generally attempted comprehensive change; moreover, several 
were among the best and most interesting Innovations we h;?ve seen. ' 
Yet change is harder to obtain and continue at the secondary school 
level-. (Table 5.1 shows that elementary school projects were more, 
likely to achieve their goals arid to produce teacher change; conr- 
tinuation of methods and materials was also somewhat more likely, • 
but not significantly so', for elementary schools.) 

For example, secondary school teachers are often characterized 
as "subject-oriented/'- in contrast to the "child-centered" orienta- 
tion attributed to elementary .teachers. Such a particularistic 
perspective is incompatible with many of the changes attempted by 
comprehensive innovations — changes, that attempt to minimize the 
distinctipns between academic disciplines. As one teacher partici- 
pant in a secondary level team-teaching project reported, "No one — 
administrators, teachers, consultants — was ever able to determine 
how to meet, the requir*^ment of writing interdisciplinary units and 
at the same- time maintain * the integrity of the individual disciplines. 
In my opinion, this was an impossibility." Or, as a superintendent 
commented regarding difficulties that had been experienced on a 
secondary career-awareness proj ect , " [High ' school ] teachers are ' 
simply unwilling to vacate- [what they see as] their responsibility 
to subject matter in adjusting to supplemeneary materials." Sec- 
ondary level projects that require a significant amount of change, 

46 ' ^ 

Because the Phase II sample pool was supplemented by projects 

not in our original sample, the total number of secondary schools in 

the Phase II sample was 29. 
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in short, apponf tO:, be part Iculnrly cl.ifficult to sustain. 



Tc.icher Charact e t ist ics . \ 

Most resLVirch on "teacher effects" (i^xploros the relationship 
between" various classroom practices and t^eacher' behavior . and student 
performance . Perhaps because the persona^l at t ribut es o f teachers do 

not appear amenable to manipulation by policy, teacher character- 

■ \ ' 48 . 

ist ics per se have not received much attention. Nonetheless, a 

"conventional wisdom" has developed concerning the effects of 

various teacher attributes: that older teachers are Jess willing 

to change, that the best ideas come from younger teachers, that 

teachers with high verbal ability are more able to achieve cognitive 

gains, and so on.* Such beliefs, in add i t/ion t o our own observations 

in classrooms, suggest that the personal characteristics of project 

teachers could have significant import for the implementation and 

longer-term outcome of federally supported innovations. Are some' 

teacher attributes significantly and consistently associated with 

"successful" projects? With continuktion of project strategies? 

To examine the relationship between teacher characteristics and 
project implementation and contlnuat ion, we collected data on sev- 
eral teacher attributes most often cited as s ign;i f icant ly affect:^g 
bot h s t udent pe r forma nee and t he out come o f innovat ive p roj ects : 
age, educational background, verbal ability, years of experience, 
and sense of^ efficacy. 

We did not use the first two in our final analysis.. Age was 
•highly correlated with experience, and where it wa,s not a proxy for 
yea^s of experience, showed no identifiable or consistent pattern 
of relationship to other variables in the analysis. i Teachers* 
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Volume II, pp. 54-55, discusses the problems of innovation 

for secondary schools. \ 

■ 4& . ' ■ ■ -i 

With some notable exceptions. See, ^.g., Jam^s S. Coleman 

et ai.', Kquali.ty of Educational ■ (yppovtuniinj ^ Office o^f Education, 



National Center for Educational Statistics, Government Printing 
Office, Washington, D.C., 19.66. Also see Eric )^3.r\us\\ek ^' Eduoati on 
and Haoe^ \). C. Heath, Lexington, Mass., 1972, and Richard Murnane, 
The Frnpact of School Resources on the Learning of Inner Citij rhil-, 
(rir5??, Bal linger, Cambridge, Mass., 1975, \ 
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educat ionnl background--t.he quality of their undt^rgradiiato institu- 
tion as spoci f led by' the Hc-hoo 1 quality index in lUirron'ji (Juidc to 
Colleges and l|niversit ies — was. not useful for the analysis, because 
of its lack of^variation for our sample; the vast majority of the 
teachers we s'lrveyed had attended state colleges i>r universities in 
the areas where] they were currently teaching, and .those i nst it ut ions 
had approximately the same q ua 11 1 V rating- on the BarronVs index. 

The three remaining .teacher attributes — years of experience, 
sense of efficacy^ and verbal abi I i ty—sign L f i can.t ly affected project 
outcomes. We found that years of experience and teacher sense of 

.efficacy had strong and s ign i f leant , . hut very different , effects on 

49 ■ 

most ol the dependent variables. Specifically, the number of years 

of teacher experience w/is t u.\u>' related to aM of the dependent 

variables with the exception of teacher, continuation of project 

materials and change in student achievement, where there was no 

significant' relationship (see Table 5.1). In other words, the more. 

.experienced a project teache r ,' the less likely was the project to 

►achieve its goals' (-.13), and the less likely was the project to . 

improve student performance (-.09). Furthermore, more, experienced 

teachers were less likely to have changed their own practices (-»15) 

and less likely to continue using project methods after the end of 

federal funding (-.07). • . 

The teacher's sense of ef f icacy--a he] ief that the teacher can 

help even the most difficult or unmotivated student s--shows a strong 

positive relationship to all of the dependent variables in our analy- 

. 5'0 ' • ' 

sis. Indeed, the regr^sion coefficients of the effects of a sense 

49 ~T~ 

Variables can have opposite effects in a multiple regression 
if they are highly correlated . However, teacher experience and a 
sense of efficacy were correlated o.nly .06 in our sample (see App.-A). 
Neither variable is highly correlated with ;any of the independent 
variables used in Tables 5.1 to 5*..3,' nor is there any significant . 
pattern of multicollinearity. Moreover, App. B shows, by means of a 
highly sensitive statistical procedure, that sense of efficacy, years 
of experience, and verbal ability were not significantly related to 
the project- or the school in which the respondents taught. 

^^Our -measure of teachers* sense of efficacy, was based on two 
questions. One -asked ' whether the teacher felt that 'Vhen it comes 
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of efflracy arc among tho strongest rela t ionships i dent 1 f i cd In our 

analysis. As Tablo .5.1 indicates, teacher sense of efficacy is 

positively related to the percent of project goals achieved , the 

amount of tonclier change, improved student performance, and continu- 

ation of both project mctliods and materials. Teachers* attitudes. 

about thei^r own professional competence, in short, appear to have 

major effects on what happens to projects and how effective they 
51 

are. . ' 

Teachers ' verba I ability, uni ike the preced ing a ft ributes , was 

significantly related to only one of the dependent variables, total 

52 

improvement in student performance (.12). However, when student 



right down to it, a teacher . real, ly can't do much [because] most of 
n student's motivation and performance depends on his or her. homo, 
envirbhment . " The other asked whether the teacher thought that "if 
I really try hard, I can get through to even the most difficult or 
unmotivated students." Responses to these two questions were com- 
bined into a single measure of efficacy — the extent to which the 
teacher believed lie or she had the capacity to affect student per- ■. 
formance. The' standard discussion of efficacy, on wlxich we** based 
pur instruments, is in J. B. Rotter, "Generalized Expectancies for 
Internal Versus External Control of Reinforcement," Psychological ' 
Monographs, Vol. 80, No. .1, 1966. . < ■ ^ ' 

^"^A Rand study of the School Preferred Reading program in Los 
Angeles drew heavily on the instrumentation and design of the. present 
"Change Agent" study arid reached similar ■ cone 1 usions . Specifically, 
it concluded that', "The more efficacious the teachers felt., the more 
their students advanced in reading achievement. This measur'e was 
strongly and significantly related to increases in reading." This 
study found no relationship between teacher background character- 
istics (race and ethnicity, college attended, undergraduate major, 
whether any graduate training was received, amount of college 
instruction in reading, and teaching experience) and students' 
achiievement . See David Armor et al . , Analui^is'' of the- School Pre- 
furred Reading Prograiri irL..Selected Los Angeles Mindrity Schoolj^ 
The Rand Corporation, R-2O07-LAIJSD , August 1976, pp. 23-24. It is 
also important to note that this Rand study used, as the dependent 
variable, the change in individual students' scores on a standardized 
read ing tes t . " • 
• 52 • ' 

Teachers' verbal ability was measured by a self-administered 
Quick Word Test consisting of a fifty-question, multiple-choice, 
vocabulary-type test . We wish to thank Harcour't Brace Jovanovich, 
Inc., for their permission to use Level II of the. Quick Word Test 
(QWT). See Edgar F. Borgatta and Raymond ^ J . Cors ini , Quick- Wo y^d 
Test, Harcourt Brace and Jovanovich, New York, 1957. We chose QOT 
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pcrfornKinco is broken, down into its cognitive and affective compn- 
nents, the data indurate* thai must of the effect of teachers '■ verba I. 
ability was on coftnitive achievement; It apparently had no signifi- 
cant effect on students' affective deveJopment (see Table 3.2). 

In summary, the teacher's sense of efficacy had a strong effect 
(Ml project continuation. Years of experience, on the other hand, 
was negatively related to most (»f the proj(.*ct outcome measures. 
Teachers' verba 1 ab i lity had no relationship to project Lmplementa- 
ti(Mi, iMit.ome, or continuation with the exception (if its pc^sitive 
forrelalion to imprc»ved student achievemt»nt , 

The [Powerful effcnt of a teacher's sense of efficacy raises thc» 
question of whether it might be possible to enhance it t h rough ' var i- 
ous project or school activities. Because we did not measurt» this 
t(>acher attribute before the project began, we cannot say whether 
prcMei't act ivit ies (such as training) mayhave changed it. Our 
impression is, however, that it reflects teachers' school experiences 
as well as their personalities, [f so, staff development strategies 
aimed at teachers and their environment might increase tlieir sense' 
i^f ..efficacy . In any event, given the important relationship between 



as a measure of verbal abilities, because It has high reliability and 
correlates highly with more complex measures of Intelligence. See 
.1. C. Nunnally, "Review of Quick Word Test," The Seoenth Mental Mens- 
■ircneyiiB Yc.ivhook^ pp. 3 78-379. 

The response rate of 93 percent for the QWT was only slighMy 
lower than the average response rate across all questions in the 
teacher survey. The distribution of responses Was . skewed toward the 
Iiigher score-s, with a mean of 42 (maximum score was 50) and a stand- ' 
ard deviation o f « 5.6. Approximately 1 percent of the respondentrs 
were excluded from analysi s because they seemed not to take the test 
seriously. The se 1 f-adm i n i s t ered. QWT is open to cheating. We used 
sevf^ral' cross-checks within the data-set to examine this possibility.. 
Fbr example, principals rated teachers on fiye dimensions, and the 
only one highly correlated with .QlfT results was. the teacher's atillty 
to "speak and write clearly and use the English language effectively." 
These chec^ks, along with comparisons with preliminary results of the 
QWT used in the Hary Income Maintenance Experience (Information sup- 
plied by private correspondence with Richard J. Murnane) , leads us 
to believe that cheating was not substant iaJ . See also Richard J.. 
Murnane, "Correcting Measurement Errors on Self-Administered Tests: 
The Case of • Verbal Ability Tests Taken by Teachers," paper presented 
at the meeting of the Econometric Society, September 1976.. 
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this toachcr attribute and. the success of innovations, further 
rosoarc-li on ways to enhance teachers* sense of efficacy would be 
appropriate. 

Sifflfl^rly, the negative effect of years of experience on imple- 
mentatitiii and continuation should be a major concern of policymakers 
Rather than continually learning and renewing themselves, many 
teachers seem to get- into a rut after their first three to five 
years of tnarh.ing. For many teachers in our sample, the passage 
of time on the Job seemed to diminish their capacity to change and 
perhaps dampen their enthusiasm for innovations. We suspect this 
unfortunate effect is loss an intrinsic characteristic of teachers 
or Che teaching "role" than an^ organ izat ionaJ- de fee t in the way 
schools manage themselves and the professional development activi- 
t ies of their staff. 

*> 

Summa ry of Effects of Institutional Setting 

In summary, a number of factors in projects' institutional set- 
ting were found to have major effects on outc .-n ^ . In general, how- 
ever, school background characteristics per sc hnd little signifi- 
cant relationship to implementation and outcome! Project-s conducted 
in wealthy schools, for exan^ple, were no more or less likely to 
succeed, all other things being equal. With the exception of the 
problems encountered b> secondary schools, the type of school 
mattered less than did the school's climate and leadership and the 
characteristics of the project staff. ' 

^Leadership was a strong influence at both the scho(M and project 
.lev9l. An effective project director was critical to successful 
implementation; principal support was important to implementation 
and especially to continuation, The quality of working relation- 
ships-, or the cliT^ate of the project, was also significantly related 
to both the percent of project goals achieved and to continuation. 

The teacher's sense of efficacy— the extent to which teachers 
felt they could reach even the most difficult or unmotivated student 
in their classroom—emerged as an important variable in. our analysis. 
It exerted major positive effects on all our outcome measures. 
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Teachers' ytMrs of cxperionrc hnd a s Ign i f i cant 'nefta t i rel.atlmi- 
ship to project succlsss. Teachers* verbal ability was related to 
improved student perfomuinre, but did not. seem to affect other pni- 
jeet outcomes. 

Tliese Institutional factors appear to be amenable to tlie tnflu.- 
ence of local planners. For example, in 'light of the great Importance 
of principals^ planners ci)uld strive to identify cand idate schoo Ls' 
where projects will be assiired o f pr Inc ipal s ' support, or devise 
strategies for eliciting princlpalr.' support before projects are 
Implemented.^ Furthermore, a number of project implementation strate- 
gies — staff training in particular — appear ' to of fer a vehicle for 
influencing the quality of staff working relationships and possibly 
even teachers' sense of efficacy. In short, our analysis suggests 
that planners and policymakers do not have to take inst I tut ional 
factors as "given," but as features subject to judicious selection 
and shaping. «. 

SU MMARY OF FINDINGS AT CLASSROOM LEVEL 

This chapter has examined the effects of three categories, of 
factors : the federal input , project characteristics , and the insti- 
tutional setting — on the effects and continuation of change agent 
projects at the classroom level. At the beginning of the chapter, 
w<B presented an overview of the findings that listed each !major 
result in turn. The following summary takes a different approach. 
From the outset, we assumed that the teacher ' s .con t inuat ion of a 
project depended on the complex interplay betpween, on the one hand, 
characteristics of the project. and Institutional setting (including 
personal characteristics) and, on the other hand, the effectiveness 
of the project's implementation and the change it produced in teach- 
ers. To provide a sense of these systemic re lationsh"fps , the remain- 
der of this chapter reviews the classroom-level findings by ordering 
the various' effects of the factors in terms of their primary influ- 
ence on each of the dependent variables. 

In particular. Figs. 5.2 and 5.3 will be used to summarize 
findings about the teachers* continuation of project methods and 
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Teachers' Verbal Ability 

NOTEf Factors marked with an asterisk. (*) had a significant efffirt^^on continuation when Percent^ 
Project Coals Achieved or Teacher Change was not controlled for". Factors marked <-) had a negativij. • 
effect; all other facrors had' positive effects... 

^Number of Project Schools had a positive relationship to continuation of project nwjthods that 
was barely slgnif leant at the ' . 10 level In Table S.3 and was not significant In Table 5.1. However, 
the strength and significance of this. effect Is reduced c;onslderably when the number of project, 
schools Is normalized for district sUe. Because" the significance of the number of 'project 'schools 
Is therefore questionable, It has not been counted above .as having a significant effect. The effect 
of number of project schools on continuation of pro jvc'^ matet*iala Is not reduced when the number of 
project schools Is normalized for district size. ^ 
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Fig, 5.2 — Factors affecting continuation o.f project methods 
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T^fc r'?r Continuation of Project Materials when Percent Project Goals Achieved or 

llTplsi^^^^^^^^^ controlled for. Factors marked (-) had a negative effect; all other factors 

Cont^iua^"^o?lro]:^ ^'^^^'^^ ^^-^-^^ ^"--^ 



Fig. S-Vp'actors affecting continuation of project materials 
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project materials , respectively . These diagrams, derived by compar- 
ing the "reduced form" coefficients in Table 5.1 with those of the 
recursive coefficients in Table 5.3, portray the system of relation*^ 
ships among both independent and dependent variables used in the 
earlier analysis. The diagrams identify., by the use of arrows, 
factor.s that had a "direct" and significant influence on each depend- 
ent variable: percent project goals ach ieved ,, teacher change, and 
continuation. An asterisk denotes a factor whose effect on pfoject 

continuation seems to have resulted from its effect on percent pro- 

53 

j ec t goa Is achieved or on teacher change . . 

(Continuation of Project Methods 

The continuation of pr^pject methods by classroom teachers 
depended primarily on how well the project was implemented during 
the period of federal funding, and especially on how much change was 
produced by the project in the style and behavior of teachers. These 
relationships are not surprising because- effective continuation, or 



The diagrams (^f.fer.., in effect, causal models based on our • 
beginning assumptions and the data analysis ;. they are interpretations, 
rather than fully tested models. Consistent with the study's explora- 
tory nature, they should be treated as hypotheses. The diagrams were 
constructed in the following way. All factors that had a "strong" 
significant effect (i.e., at the .05 level) on continuation (control- 
ling for percent goals achieved and teacher change) are joined* to 
continuation by a direct arrow. The di^agram reflects some subjective 
judgments. For example, the int roduc t ior^ of percent goals achieved 
into the regression causes a drop in the effect of "teachers work 
together" on continuation of project methods from a highly signifi- 
cant .13 in Table 5.1 to a barely significant, (at the .10 level) 
effect of .09 in Table 5.3; moreover, when "difficulty of implementa- 
ti .\" was also taken into account in preliminary analyses, the coeffi- 
Ci.Ti of "teachers work together" dropped even further to an insif.'tJ- 
fiv c .06. Considering the strong .22 effect of "teachers work 
together" on percent project goals, it made sense to interpret thr 
effect of "teachers work together" as being a direct one on imple- 
mentation but an Indirect one on continuation of methods. Factor^ 
having direct significant effects on either percent project goals ':ir 
teacher change are shown by arrows. Some of the judgments implied 
in these diagrams required a variety of preliminary statistical pro- 
cedures and tests, particularly F-tests. In keeping with the summary 
and exploratory nature of this section, the results of. the statistical 
tests are not discussed. 
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assimilation, means that teachers must learr, project approaches and 
integrate '^hem into their i:lassr()om routine. Nonetheless, it is 
important to note r.hat the evidence strongly supports the study's 
beginning hypotheses: effective implementation and teacher change 
were essential to continued use of an innovation. Factors that 
contributed to implementation and teacher change could also be 
thought to have indirectly promoted project continuation. 

VJhat influenced implementation? ImpJ ementation, i.e.,. the 
percent goals achieved, was promoted by projects having specific 
goals (or, more particularly, by the clarity with which teachers' 
perceived project goals and operations), by well-executed training, 
by an effective' project director,, and by g( od working relationships 
among the staff. In addition, effective implementation wus more 
likely in elementary schools and in larger schools. All these vari- 
ables, shown on the left side of Fig.. 5.2, exerted their primary 
effects on implementation, but implementation was additionally 
influenced by factors that also contributed to other project out- 
comes. Thus, Fig. 5.2 Indicates that teacher participation in pro- 
ject decisions and principal support promotred both effective Imple- 
mentation and. continuation of project methods. 

Teacher participation had a critical instrumental value for 
implementation, achievement of project goals, and continuation of 



Ihis summary does not review the findings for improvement in 
student performance. However, the effect's of imple'mentation. and 
teacher change on student performance were not the same as their 
effects on confzinuat ion. For continuation of project methods, 
teacher change had a much .stronger effect (.3/i) than did percent 
goals aciiieved (.17), as Table 5.3 indicates. This pattern was 
reversed for student performance (as well as continuation of project 
materials), with goals achieved having a .27 effect and teacher 
change having a .18 effect. These, data point to the dilemma dis- . 
cussed in the text. Namely, improvements in student performance 
can be enhanced in the short run by putting resources into effective 
project implementation (particularly if the proj ec t . includes skill- 
specific staff training and has efficacious^ teachers); yet, the con- 
tinuation of such improvement for successive generations of students 
may requird a greater emphasis on promoting teacher change as well 
as those activities (e.g., staff support) that enhance the teacher^s 
learning of new methods. 
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project methods. We found that teacher suggestions can help a pro- 
ject turn, out ..etter, and ' their part ir:ipat ion in project decisions 
can give tht^m a sense of "oivmership" that greatly improves the 
chances for continuation of project methods; 

The principal's influence, as expected, strongjy affected . 
. implementation and continuation at the classroom lev?l. The princi- 
pal's unique contribution to implementation seemed ':;o.lie in giving 
moral support to i*the staff and in creating an organizational climate 
that gave the project "legitimacy." The principal's support seemed 
an almost necessary condition for project continuation. This rela- 
tionship is understandable, seeing that the principal typically sets 
the educational style of the School. Teachers are unlikely to con- 
tinue a full array of project methods without the sanction of their 
principal, even if the methods were successful and had been assimi- 
lated. To do so would not only be difficult in light of the some- 
times subtle, sometimes blunt means that principals often employ to 
establish a uniform "school style," but also would appear contrary 
to professional self-it|terest . Principal support, then, V^ad a major 
effect on continuation that was independent of project activities 
or outcomes. The principal amply merits the title of "gatekeeper 
of change." 

Teacher characteristics affected both implementation and the 
extent, of teacher change, as Fig. 5.2 suggests. More accurately, 
they defined the parameters within which project outcomes became 
possible: Efficacious teachers were associated with more effectively 
implemented projects and with more project-related teacher change; 
more experienced teachers seemed to be less flexible^during imple- 
mentation and Jess susceptible to change. 

The remaining factor that promoted implementation was staff 
support strategies; Fig.. 5.2 shows they also. made major contributions 

^^Though teacher characteristics did not have strong, direct 
effects on project continuation controlling for implementation and 
teacher change, they had powerful effects on student outcomes : The 
more efficacious teachers were and the higher their verbal ability, 
the greater was the improvement in student performance (see Table 5.3). 
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to teacher change and project continuation.- They were so central to 
all project outcomes, in fact, that a summary of their role, is best 
deferred until after wereview variables that promoted teacher change 

. : What factors influenced the extent of teacher change,. in addi- 
tion to (i.e. , Controlling for) teacher characteristics? The more 
effectively the project was implemented, the greater was the extent 
of teacher change. Though this relationship was expected, its magni- 
tude was lower than anticipated. Whereas poorly implemented projects 
werc^ unlikoly to result in teacher change, the amount of change for 
effect ivcly implemented projects apparently depended on other factors 
In particular, the right-hand side of Fig. 5.2 indicates that the 
substance and scope of the innovation had major effects on the extent 
to which teachers changed their style's or classroom approaches. , 
Thus,- complex innovations, such as classroom organization change or. 
projects focusing on parent involvement, were more llkelv to promote 
teacher change than were other, types of projects, even when the 
effectiveness of the project's imnlcmentat ion had been taken into 
account. Similarly, innovations requiring change in the overall 
teaching style of project staff and calling for extra effort resulted 
in more teacher change. 

Both teacher change and implementation were influenced by another 
implementation .St rategy — a factor that by itself appears to be a *'key 
policy lever'* to the continuation of project methods: well-executed 
staff support strategies. The project activities that accompany 
training — i.e., classroom assistance, regular meetings, usefuT out- 
side consiiltants , observat-ion in other classrooms — had, as a group, 
strong significant effects on the percent of project goals av tiieved, 
on the amount of teacher change, and on the likeJihood tha t, p.'oj ec t 
methods would be continued. (Skill-specific training, in cohl. .;r>t, 
had an indirect effect on continuation of methods, through its posi- 
tive relationship to the percent of project goals achieved. If 

^^Well-executed training of teachers directly improved student 
outcomes, but not continuation of project method;:; and materials (sec 
Table 5.3). 
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continuation of project methods is seen as asslii.ilation , the impor- 
tance of this factd: becomes clear. Assimilation required that pro- 
ject approaches-' che Information conveyed through training sessions, 
for exainplo- -Oc individualized for each teacher and integrated into 
classroom practices. Implementation support activities that were 
conducted in a practical (or concrete) and timely manner provided' 
the means wliefeby teachers received feedback and .advice they needed 
to make the project work for them in their classroom.' In short, 
strategies that supported- mutual adaptation thereby promoted imple- 
mentation, teacher change, and continuation of project methods. 

Continuation of Project Materials 

Figure 5.3 depicts teacher continuation of project materials as 
being directly infllienced by more factors than was teacher continua- 
tion of project methods. That is, such factors as funding level, 
number of project schools, or specificity of project goals directly 
affected continuation of project materials regardless of (i.e., con- 
trolling for) their effects on implementation and teacher change; 
the same relationships did not hold for continuation of project 
methods. This suggests that teacher continuation of proj ec t . mat er i- 
als not only was a somewhat different process but also that it was 
easier for teachers to continue to use project materials after the 
end of federal funding. 

Specifically, the continuation of project methods. of ten repre- 
sented or required a fairly comprehensive change in classroom 
practices, whereas the continuation of project materials did not. 
Project materials — a new reading syllabus, for example — can be inte-^ 
grated into classroom practices without any change in fundamental 
classroo^ii procedures. On this point, then, it is not- surprising 
that principal support had a weaker relationship to continuation of 
iraterials than it had to continuation of project methods. 

The extent of continuation of project materials depended more 
on the substance and design of the project than did the continuation 
of project methods. In particular, continuation of project materials 
was negatively related to the size of the. federal grant and to chainge 
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in classroom organization, but was positively related to the number 
of project schools. However, neither the expensive projects nor the^^ 
classroom organization projects placed much emphasis on materials; 
the former typically allocated a" large portion of their funds' for 
classroom aides while the latter concentrated primarily on methods 
(e.g., open classrooms, differentiated staffing) and regarded mate- 
rials as only incidental or instrumental to project objectives. In 
contrast, projects that were located in a relatively large number 
of district schools usually achieved this broad coverage "through 
dispersion of hardware or teaching materials. Indeed, the district*^' 
objective in many projects that encompassed a large number of schools 
appeared to "be to bring these schools "up to date ' in terms of cdu- 
Ccitional technology." 

Specificity of project goals had a strdnger'and more direct 
effect on continuation of materials than it did on continuation of 
methods. This. difference is understandable because of two aspects 
of specificity. On the one hand, teachers needed clarity about pro- 
ject precepts and procedures if they were to assimilate an innova- 
tion and, thus, to continue the pro j ec t ' s me thods ; because this form 
of specificity, clarity, can be achieved only during implementation, 
its effect on continuation of methods was indirect, depending on 
how effectively ^the project was implemented. On the other hand, the 
continuariaiHi r.f project materials required specificity in .a program- 
matic siMiSfj; unless teachers knew specifically how project materials 
fit into 'jlie design of projects involving curriculum revisions, for 
example, they, were unlikely to use the materials after the end of 
federal funding.. Specificity in th^ programmatic seftse is therefore 
less something to be acquired during implementation than it .is a 
:characterist ic of the innovation and its initial design. 

Most of the remaining factors shown in Fig. 5.3 had similar 
effects on continuation of both project materials and methods. For 
example, teacher participation in project decisions directly promoted 
the continuation of project materials, perhaps because of the "owner- 
ship'.' teachers felt concerning materials they helped to develop as 
part of project implementation. Finally, well-executed staff . support " 
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strategies exerted a central influence on the continuation of project 
materialis, as they did on project methods. 

. In summary, continuation of project, materials depended more on 
project design factors — e.g., the number of project schools and the 
project emphasis or focus on new materials — than did the continuation 
of. project methods.. Furthermore , continuation of project materials . 
appeared to depend much 1 ess on the principal's support. Project 
materials, unlike project miethods,, did not necessarily reflect, a 
fundamental change in classroom practices or a particular pedagogical 
point of view, and could be smoothly integrated into ongoing class- 
room routines. Nonetheless^ well-executed staff support activities 
were required to promote the adaptation and integration of. project 
materials. 

In contrast., continuation of project methods was harder to 
achieve. It required both ^ effect ive implementation and an innova- 
tion that promoted teacher change. Teacher participation, support 
from the principal, and staff support strategies that promoted mutual 
adaptation all played critical roles, directly as well as indirectly, 
in helping teachers to assimilate new practices. Consequently, 
these three factors should be the prime concern of policymakers 
seeking to promote long-term educational reform. 
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- Chapter 6 
DISTRICT LEVEL ANALYSIS 



Thus far we have examined factors that affected assimilation and 
continuation of project methods and materials by classroom teachers. 
But assimilation is only part of the picture of the fate of aproject; 
the other part consists of district-level decisions and activities. 
How and why do districts decide to continue or discontinue a change 
agent project? What happens to "continued" innovations? Are they 
still treated as "special projects" or are they incorporated into the 
district's standard operating procedure? What strategies promote in- 
corporation? This chapter first treats these . district-level questions 
and then draws together classroom and district considerations to offer 
an overall assessment of the fate of change agent projects after the 
end of federal funding. 

MEASURING CONTINUATION STATUS 

When federal funding ends, the district must decide whether to; 
continue the project, and what level of support to give it if it is 
continued. It is difficult to measure levels of support in com- 
parableterms because of the yariation in types of projects, their 
settings, and the kinds of support hey require. For example, a proj- 
ect relying on paraprof essional staff and one involving romputer- 
assisted in5--' -".ion will probably require, different kinds -of support. 
For the sal* ot st'cistical analysis, we had to code these diverse 
decisions/^in to comparable, albeit crude, categories — namely, t\\e proj- 
ect was discontinued, reduced, maintained at about the same level as 
the last year of federal funding, or expanded beyond the federal funding 
level.. 

In Phase I we asked superintendents to predict the fate of their 

. ■ r ^ 

projects. For Title III projects, 21 percent predicted expansion; 
22. percent expected . continuation at about the same level; AA percent 
indicated the project would be cut back; and^^lS percent believed it 
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would not be contiuued at all. We obtained comparable measurements 
of the status of these projects two years after the end of fejderal 
: funding by piecing together a variety of district continuation deci- 
* sions (e.g., the change in number of projecc schools, in the number 
of teaching and paraprof essional staff, and in financial support). 
Table 6.1 compBres the superintendent's estimate of expected continua— 
. tion with our estimate of continuation. . . 

Superintendents appear to have been optimistic. We found less 
continuation than they had. expected . But they were accurate in their 
"rank ordering" of projects. That is , all the projects discontinued 
at the end of two years had been expected to be either discontinued 
or cut back; all the- projects actually cut back had been expected to 
be cut back or remain at their federal grant level; etc. Assuming 
superintendents were gene rally s in.cere in their original estimates, 
the decay in the continuation status of projects after two years at- 
tests to the vulnerability of "continued" projects. 

Projects had their scope of activities reduced in a variety of 
ways. Some reductions amounted to an adjustment of the project: after 
the end of federal funding; other reductions were true "cut backs" 
and were coded accordingly. For example, the number of project schools 
decreased in some districts — usually, according to the superintendents, 
because of lack of interest of the staff at; particular schools, although 
they also mentioned budget difficulties. When the original teaching 
staff for a project was cut back sharply, we coded the loss as a reduc- 
tion in the project. Personnel other than classroom teachers were 

2 

likely to be reduced, even on projects that were otherwise secure; 

"^Since the original sample was not representative , these percent- 
ages should not be taken as estimates of the population of all former 
Title III projects. . . 

2 

The distribution of changes in support personnel for 98 of 100 
Phase II projects was: - 

'Stayed - - , 

Increased Same Decreased Elimina ted 

Training personnel or 
specialists other than 

classroom teachers ' 13% 32% 33% 22%. 

Aides or paraprof essional 

staff 12% 26% 43% . . 19% 
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Continuation Status 



Table 6.1 
CONTINUATION STATUS OF PROJECTS 



n Origina I Sample 

Original Sample Measured About 

Measured in Two Years After 
Last Year of End of 

Federal Grant. . Federal Grant 



Expanded * 
Maintained at federal 

grant level 
Reduced 
Di scontinued 
■> ■ 

Number of responses 
Not codahle 



21% 

22% 
4A% 
13% 

17 3 : 

21 



11% 

, 21% 
23% 
45% 

158 
A6 



Phase II 
Sample 
Excluding 
Discontinued 
Projects 

13% 

35% 
52% 



96 
4 



NOTE: The first column represents the responses of superintendents 
to the following question: Do you expect this py*ojcab ioill be expanded., 
cwitinued at the present levels out baak^ or not . continued at all after 
tcrmi?Latio?i of federal funds'/ 

■ The second and third columns represent a coding from a variety of 
different questions in the Phase II telephone survey and superintendent's 
and principalis questionnaires. The sample for the third- column excludes 
71 discontinued projects and 35 nonclassroom projects from the original 
sample but includes 12 former Title III projects not in the original 
sample. / 



we did not consider these projects to have- been cut back unless the loss 
of support personnel appeared to have 'Seriously hampered the project 
(for example, the parent-involvement component of one project relied on 
home visits by paid aides, who were not supported by the district after 
the end of f-..ieral funding) Most of the projects had their budgets 
trimmed or elin-inated; 35 percent received - con tinued support from sup- 
•plementary funds, primarily from the local school district budget: over 
one-third of the projects in our unusual sample received state or federal 
funds for the dissemination; of the proj ect outside the district. We con- 
sidered projects to be cut back in their funding only if other evidence 
in the inlervlews indicated that they had financial difficulties. 

In summary, at the end of federal funding district officials de- 
cide how much and what kind of support to give a project." For the sake 
of statistical analysis, we coded this continuation decision into the 
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four categories shown in Table 6.1, and thus constructed a dependent 
variable. In addition to questions about the reliability of our 
coding, the interpretation of this variable is not straightforward. 
For example, is a project that has been cjt back less likely to be 
sustained than a project maintained at the same level? Or, do teachers 
use a project that has been expanded extensively or less? Questions 
of this type will be considered sub&^equently . Nonetheless, the reader 
should view the following analysis with the same caution as the authors: 
Its purpose is exploratory and the findings are no more than hypotheses. 

FACTORS AFFE CTING THE CONTINUATION DECISION . ■■ 

In contrast to the classroom-level analysis, one could list a 
multitude of possible f^actors that impinge on districts' continuation 
decisions. At the classroom level, we were examining forces impinging 
on individual teachers; despite the lack of a we il-developed theory, we 
could identify major factors derived ' f rom ' our empirically grounded con- 
ception of the local process of change. At the district level, many 
people with diverse agenda are engaged in a complex interplay that 
subjects the decitjion to a host'of idiosyncratic situations and influ- 
ences. Instead of attempting to, account for all these particularities, 
we used a simple conceptualization of the decision based. on the Phase I 
research. 

We conceive of the district's decision on continuation to be a 
function of four general factors: the likelihood of the project's 
future educational effectiveness, the importance of the educational 
need it serves, the resources it requires, and the organizational and 
political forces that inhibit or promote it. 

Table 6.2.presents the results of a multivariate statistical 



I We assumed these categories were ordered and assigned an interval 
scale to them..^ A s tat is tical J y superior but very costly procedure 
would have been to consider the categories as being nominally defined 
and to have used a series of dichotomous or polychotomous probit analyst,^ 
The crudity of our measurements, as well as the exploratory nature of the 
analysis, did not justify such expenditures. We also divided the data 
into four subsamples corresponding to the four categories of continuation 
' status and compafred them in terms of characteris't ics of various indepen- 
dent variables.. These comparisons provided' evidence for some conclusions 
discussed in the next section. 

• . . '176 
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Table 6.2 



FACTORS AFFECTING CONTINUATION STATUS 



Standardized Regression 
Coefficient for 
Continuation Status 



Model I: 
Includes 
Perceived 
Educationa 1 



Model II: 
Excludes 
Pe rceived 
Educational 



Explanatory Fiic. tor 

Perceived educational effectiveness 

Generality of the innovation 

Ptoject resource allocation 

Funding level during grant period (log) 

Funding per student 

Number of project schools 

Project covers elem. and sec. schools 

Adequacy of district finances 

Organizational forces 

Initiate project for outside funds 
Initiate . project if can be continued 
Project supported by principals 
Superintendent's tenure 

Local political considerations 

Extent of political-social difficulties 
Difficulty from' student test scores 
Percentage of population white 

<■ School district enrollment (log) 



Effectiveness'^ Ef f ectiveness 



38' 



,22 



.16 
.28 
.12 
.26 
.02 



.18 
.24^ 
,18 
.18 



-.04 
.24^ 

-.06: 

. 22^ 



.34 



-.17 
-.29 
''.13 
-.28 
.03 



•k-k 



-.19. 
.22 
.22 
.15 



-.06 
.20^^ 

-.10 
.24"* 



R^ . 
Number of observations 



.44 
88 



.49 

88 



The estimating procedure for Model I is two-stage least squares with 

"perceived educational effectiveness*' as an endogenous variable. The 

estimating -procedure for Model II is ordliiary least squares. 
J. 

Significant at. the .10 level; 
Significant at the .05. level. 



analysis relating the continuation 'status 5f the project (two years 
after the end of . federal fundin'g) to operatidnal me.asurements of these 
factors. The remainder of this section interprets the effects of each'^ 
factor in turn.. ' „ " 

P erceived Educational Effectiveness dr Piroject "Success " 

Virtually all federally funded Innovations pr.o'duc6 evaluation docu- 
ments that report on the project ' s' "success . " But these evaluations 
typically bear little resemblance to what really happened, on the project 
Moreover, regardless of their accuracy, school district officials sel^ ^ 
dom take these reports seriously. ^-Ihstead.',^ through formal. and informal" 
discussio.ais with project participants, they develop a "feeling" for how 
effectively -the project was,, implemented and thereby evaluate the proj- 
ect's potential. , ■ . ' • 

We used a' statistical procedure-tb measure this evaluation. Using 
data collected in Phase I, we considered the superintendent's report on 
the^ percentage of the project's goals achieved to be =a function of 
principals' and teachers' perceptions of implementation outcomes. Be- 
cause more than one. principal , as well as several teachers, could have 
been involved in an innovative project, we assumed that the superinten- 
dent integrated their responses — which sometimes disagree^d--by averaging 
the principals' responses, averaging the teachers' responses', and 
weighing the disagreement between people at the same level. W[e call 



The dependent variable . used in Table 6.2 is the continuatiph 
status of the project two years after the end of federal funding. . 
Table 6.1* shows the distribution of this variable". The independent 
variables . were measured in the last year of federal, funding; Ghap. .5, 
Vol. II, describes their measurements in more detail. 

The statistical procedure whose results are shown by the first 
column of table 6.2 is two-stage least squares. The first stage was 
used to estimat.e the perceived educational effectiveness of the project, 
as described in the ne\t section. R2*and the t-statistic for this pro- 
cedure have somewhat different meanings from those in ordinary least- 
squares regression. See P. Dhrymes, Eoonometrias ^ Harper and Row, New 
York, 1970. Because the estimates from the two-sta'ge procedure are 
sensitive to our assumption about how to calculate the project's per- 
ceived educational effectiveness, the second column of- Table 6.2 
presents regression results using ordinary least squares, without T 
perceived educational effectiveness. 
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the resulting measure the estimated educational effectiveness of the 
project, as perceivied /by the superintendent, or, more simply,, its per- 
ceived success.^ 

The reader may infer from Table 6.2 that perceived success had a 
positive effect on continuation status. Much of this effect, however, 
can be attributed to discontinuation. (A comparison of the four sub- 
samples, using analysis of variance techniques, shows that the average 
perceived success of discontinued-projiects was significantly less than 
the average perceived success of the other three groujps'^ take:> sepa- . 
rately or combined.) That is, projects poorly implemented or hardly 
implemented at all were usually dropped. - This finding ir-, uot sur- 
-pirising, but it is important to realize that perceived success did not 
determine continuation status to a greater, extent. Indeed , the data 



5 -n ' ■ 

Statistically, we created the estimated educational effectiveness ^ 
measure by estimating the follofwin| equation: Percent project goals 
achieved according to superintendent- = a . , * 

+ b^ (avg. princ. % goals achieved) + b^ (avg. princ. difficulty of impl.) 

+ b^ (avg.. princ. impl. as laid out) + b^ (var. princ. % goals achieved) 

+ b^ (var. princ. difficulty of impl.) + b^ (var. » princ. impl, as laid out) 

+ b^ (avg. teacher and goals achieved) „+ bg (avg. teacher change) 

+ (avg. teacher difficulty of impl.) + ^iq (avg. teacher impl. as laid oi^d) 

+ ^11 (t^a^^er variance % goals achieved) • - 

+ b^2 (teacher variance rteacher change) 

+ h^^ (teacher variance difficulty of impl.) " ' 

^ ^14 ^teacher variance impl. as laid out) " . 

This equation formS; the first stage of a two-stage least-squares estima- 
tion in which the above independent variables are instruments . and educa-* 
tional effectiveness is endogenous. See Chap. 5, Vol. II. 

Ins^^ad of this complex procedure for estimating perceived success, 
we could have simply used the superintendent's answer, but that answer 
may be inextricably related to the superintendent's view^ of the project's 
centrality, (as well as to his or her own char.acteristicsK Therefore, 
.'we would introduce statistical 'errors (a simultaneity bias) into the " 
: analysis if we used ,both variables as explanatory variables for a proj- 
: •. ect's' continuation. Instead, our "measurement" is an estimate of t/ie 
superintendent's answer, about success, based on variables that cao /be 
• assumed to be unrelated (or exoge^nous) to the super^tendent ' s per^ep- 
/ tion of centrality or his or her personal character istics . 
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show that, on the one hand; a number of less "successful" projects 
were expanded and, on tha other hand, numerous "successful" projects 
were reduced at the end of federal funding. In c .ler words, if a 
project was not dropped, then its cont :i ^n status — and perhaps the 
.level of support from the district — de on factors other than its 

perceived suv^cess. 

C&ntrality : In. ^^^t Wan^e of the Educational Need the Project Serve d 

Our origins'! sample contained a number of Title III projects that 
did not appear to be central to .the primary educational needs of the 
dlptrict. For example, one project consisted of field trips to the 
local zoo. Such projects may achieve a high percentage of their goals 
(the zoo project did)," but the evidence in Table 6.2 suggests chat they 
were unlikely to be continued. 

r particular, the Phase I* survey asked superintendents to rate 
r.he project in terms of how, close its goals were to the district's major 
educational objectives.-^ We used the responses as a crude operational 
measure of- centrality. Tabic 6.2 indicates that the more central the 
project, the more likely its continuation. This effect of centrality 
results in part from the positive correlation between centrality and 

^The specific question a«ked .of superintendents in the Phase I 
survey and the frequency of their responses for Title III projects were: 

(Centrality) How would you rate this project in terms of how nlose 
its goals are to- the major eduoational^ objectives of this 'district? 
Would you say very close , moderate Ly close ^ or not very close? 



Very close -. . . 69% 

Moderately close 29% 

Not very close 2% 

Number; of answers was 112. 



The Phase II survey asked the question somewhat differently with 
the following distribut ion of 96 response s : 

(Centrality) To what extent would you say this project was intended 
to deal with the major* educational problems of this district? 

35% 32% 17% 6% 5% ♦ 5% 0%. 

' L— J -^1 I I _i 

7 .6 5. 4 . 3 21. 

To a very Not at all 

great extent 

' . . ISO / 



perceived success — i.e., ancillary projects were likely to be poorly • 
implemented and not considered succossful./ But centrality still had 
a significant, positive effect even when success is controlled for, 
as it is in column 1 of Table 6.2. The reason is that insofar as an- 
cillary projects were continued, they were particularly prone to being 
reduced. For example, np cases in our sample that superintendents con- 
sidered ancillary (i.e., they were not rated as being "very close" to 
the district's educational objectives) were given any supplementary 
district funds or assigned any central office staff after the termina- 
tion of their federal grant. 

. ' ^ 

Project Resource Allocation > . .. 

The next group of factors listed in Table 6.2 involve the project 
resources during the peri^ . of federal funding. Chapter 5 discussed 
the effect of these variables on project outcomes and continuation at 
the classroom level. We found that, for our sample, the three variables — 
funding level, funding per student, and number of project schools — did 
not have major effects on classroom outcomes or continuation. (The ex- 
ceptions to this general finding werie: (1) More concentrated funding 
was positively associated with s tudent , improvement , and (2) " the instal- 
lation of p'^ojects in more schools was positively associated? with the 
teacher's continuation of project materials.) Table 6.2 suggests that 
these ' r>urce allocation variables had a greater effect on the continua- 
tion decision, at the district level. 

. The project's funding level during the period of "its Title III 
grant had, a negative but not quite significant effect on its continua- 
tion. Specifically, the more expensive the project was, the more 
likely it was to be cut back when "soft" money had run out. Title LIl 
proiects with a high funding per student were likely to be reduced, 
particularly in terms of a decrease or elimination of aides to the 
teaching staff .V Projects that spanned both elementary and secondary 
schools were likely to be discontinued or reduced to a less inclusive 



Notice that the effect of centrality of continuation is diminished 
when perceived success is included as an explanatory variable in Table 6 . 2 
(compare Model I with Model II) . 
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project — e.g., to only one junior high school. These comprehensive 

innovations not only spread iiheir financial resources' thin, but they 

8 

also seemed to-be trying to accomplish too much too soon. ' In con- 
trast to the innovations that spanned grade levels, projects located 
. in more than one or two district schools generally had a firm early 
commitment from district officials: They were rarely discontinued, 

0 Q 

though some were reduced in their project support activities. 

The. next factor in Table 6.2 is not a project resource variable 
but a measure of .the district's financial situation. We asked super- 
intendents in both the .973-'!l974 Phase I survey and the 1975-1976 
Phasell survey to give us their assessment of the district's current 
financial situation. Though district finances generally^ seemed to 
worsen between the two surveys, neither significantly affected the 
continuation status of the former Title III proj ects . "^'^ . In othei* words 



8 

See Vol. II, Chap. 4 for a discussion of these comprehensive 
innovations . 
9 

Table 6.2 shows that the regression coefficient for number of 
project schools is positive but not significant. The data seem to re- 
flect two counteracting tendencies. On the one hand, the projects 
covering a number of schools — mostly curriculum development iimovawions 
seemed to have resulted from official school district policy and thus 
were incorporated into the standard operations of the district. For 
this reason, they were not likely to be discontinued. On the other 
hand, these projects tended to reduce vital staff support fur tions. 
In short, they experienced difficulties in being sustained,^ as the next 
section discusses- 

■^^The following frequency of responses was given in the Phase T 
and Phase II surveys: 

HoU' do you view the present financial situation in your distriai? 

Phase I Sample Phase . II. Sample 

Only Projects Only Projects 

All, in Phase II All in Phase I 

. Finances Projects Sample Projects Sample 

More than adequate 25% 22% 2% 2:i 

Adequate 27% 24% 30% 27% 

Barely adequate 41% 45% 35% 39% 

Inadequate 7% 9% 33% 31% • 

No. of responses 182 83 97 83 

Table 6.2 uses the data of column 1, but regressions using the data of 
column 3 also show an insignificant effect* 
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relatively poor districts were no more likely to discontinue projects 
than were adequately financed districts. 

The foregoing should not be taken to mean that money does not 
matter. It does, as the next section discusses . ,,,Krrt]^er, this finding 
is consistent with our field observation that districts v.:ar) juggle the 
budgeting of their educational program in many ways according to their 
organizational and political priorities. 

In summary, the end of federal funding resulted in a downward • 
adjustment in the allocation of resources for many projects, particu- - 
lar.ly the more expensive ones. Innovations that had used soft money 
to. reduce the student-to-adult ratio (e.g.., by'hiring aides) had to be 
cut back to live within the district budget; projepts that had spread 
their resources thinly across grade levels had to be eliminated or 
severely reduced in scope. These findings, which seem predictable, 
raise the disturbing quefition of whether . school people were' unaware of 
the potential financial difficulties before they sought * a Title III 
grant. Chapter 5 suggested that the Title III program had in fact 
stimulated local innovations by providing an. opportunity for LEAs to 
use "uncommitted" money; ^hese results about district-level continuation 
decisions raise doubts about how wisely LEAs alio vd thj s . temporary 
money, even when viewed from the standpoint of t:^ school system's own 
self-interest. Does the acceptance of temporary federal "seed money" 
imply a built-in continuation problem? The analysis in this section 
scratches the surface of this issue; the next section will explore it 
more deeply . ^ . • 

Qy_Ea"i' zational Forces 

The next cluster ot variables in Table 6-2 pertain to local orga. * 
zat..onal considerations. Our field Investigations convinced us tiiat the 
bureaucratic patterns of school systems have pervasive effects on the 
fatrz- olc innovations. But those patterns ere hard to measure in compa- 
rable ways across many different types of school system^. For the sake 
of the present exploratory analysis, we selected several variables to 
represent" critical organizational ^characteristics. 

The first two variables concern the district's attitude towarcf 



specially funded projects, or the motivation underlying project adop- 
tion. In both the Phase I and Phasa.II surveys, we asked superinten- 

i ■ ■ 

dents to tell us the extent to which various considerations were 
involved in their decision to adopt a project. (Table 6.3 lists the 
specific questions asked in the Phase II survey.) From our fieldwork 
in Phase I, we hypothesized that LEA officials typically had one of two 
general attitudes in seeking a federal grant: at one extreme, ah 
opportunistic attitude in which special projects were treated as add- 
ons, to be initiated only if resources from outside the district were 
available; or at the oCKer extreme, a problem- solving attitude in which 
funding was sought for projects that were intended to attack a priority 
need and were slated for continuation at the time of the initial grant. 
We were impressed in our field visits not only by the apparent importance 
of these attitudes hut also by the way they seemed to persist, regardless 
of what happened in the classroom. Despite our inability to measure 
these attitudes with precision, the findings of Table 6.2 tend to rein- 
force these fieldwork impressions. 

In particular, the variable called "Initiate Project for Outside 
Funds" — ;our surrogate for an opportunistic attitude — affected continua- 
tion negatively, though not quite significantly. . The variable called 
"Initiate Project if Can be Continued" — our operational measure of the 
problem-sol\ ng attitude — affected continuation positively. (Table 6.3 
gives the wording of these questions.) Even if our interpretation of 
these variables as opportunism versus problem-solving is faulty, the 
statistical evidence adds credence to two hypotheses: First, initial 
attitudes toward the project mattered for Itr* continuation; second,- a 
project v:as more llV-^ly to be continued if LEA officials intended from 
the outset to iricorporate it. 

In addition to the initial orientation of the central office staff, 
the principals' attitudes toward a project affected its continuation, 
as Table 6.2 indicr <: -s . "^"^ Considering the bureaucratl^c role of principals 

"^■^It is important to note that the measure of principal support 
described in Chap. 5 is based on an aggregation of teacher ratings, viz,, 
the mean across schools of the mean of teachers within a school. Be- 
cause this variable was not created from answers given by superintendents. 
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^'the gatekeepers of change" — this finding was expected, but two dif- 
ferent organizational dynamics may have contributed to this effect. 

Some reductions in projects consisted of a drop in the number' of 
schools continuing them. Most -of these projects (54 percent in our 
sample) were originally started by district officials who had "asked" 
schools to participate. At the end of federal funding, however, some 
districts no longer' recruited schools. Instead, they often seemed to 
take the laissez-faire attitude of letting the principals decide the 
fate of the project within their schools. Some principals, especially 
those who had not been at the school at the beginning of the innovation, 
chose to let even "successful" projects wi-ther away. 

Principals* support also could enhance the project's continuation. 
In some instances there was a ."lighthouse" effect. The project . spread 
to other schools because of its prior "success'' in the district. The 
possibility of such "spontaneous" diffusion is often talked about by 
policymakers and practitioners. For our .sample, however, diffusion was 
inf requent — it was mentioned by supe r.i ntendents as a reason for the 
project's expansion in only four cases — and it was not spontaneous: 
Committed project principals actively proselytized their colleagues. 

Whereas it was relatively easy to trace the principal's influence 
cw curt iniiat ion , we found it diff icult to aiialyze the superintendent's 
iniiuence. The variation among superintendents In critical character- 
istics (e.g., whether they were "insiders" or "outsiders," whether they 
focused on politics or on internal operations, and the kind of "cabinet 
members" they chose) was too great to capture with a small sample of 
districts. Moreover, it was hard to separate the effects of personal 
traits from those induced by such district characteris Lies as its size 
and urbanization. Nonetheless, both this analysis and the cl assroom-level 



it reduces chances of spurious correlations with other questions answered 
by the super intendent~e . g. , centrality. However, this procedure may 
have artificially increased the correlation between principal support 
and the project's perceived success, which was aggregated from teacher 
and principal answers. Thus,. the regression coefficient for "Project 
Supported by Principal" is .18, which just fails to be significant at 
the .10 level in Model I. However, it is significant in Model 11, where 
the effects of "Perceived Educational Effectiveness" are not included. 
The positive correlation of -.28 between the two variables accounts for 
this discrepancy. 
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Table 6.3 
QUESTIONS ON THE PROJECT'S ORIGINS 

I'm going to read a list of Qonsiderations . ■ Using the scale on the oard^ how 
important was each of these considerations in the dist7^iat' d decision to sub- 
mit (NAME OF PROJECT) for Title III funding? First^ school board interests: 
Was tlia-t extremely important^ not at all important y or somewhere in between? 
Just, give rrie the number that comes closest to how you feel. 

Not at Number 
Extremely All of 

Important Important Responses^ 



School board interests 


21% 


16% 


21% 


26% 


10% 


■ 3% 


3% 


93 


Cbmmunity demands 


12 


• 13 


17 


22 


18 


13 


5 


94 


State or federal 


















agency suggestions 


3 


il 


18 


23 


11 


15 


19 


92 


Staff interests 


AO 


A2 


10 










94 


Need to remedy a deficiency 


















in educational pra.:tices 


53 


28 


8 


5 


1 


3 


1 


94 


Staff development concerns 


33 


36 


19 .. 


5 


3 


1 


2 


93 


Desire to experiment 


16 


20 


21 


16 


9 


6 


12 


94 


*Opportunity to obtain 


















. outside funds 


•28 


20 


16 


18 


8 


3 


6 


92 


*Likelihood that the project 


















could be continued within 


















regular district budget 


17 


36. 


2j 


10 


5 


h 


h 


92 


Prior experience with 


















the project's methods 


lo' 


10 


19 


23 


10 


lA 


15 


94 


Heard that the project 


















worked elsewhere 


5% 
1 


7% 
1 


9% 
■ 1 


13% 
1 


9% 
1 


17% 
1 


A0% 
' 1 


94 




7 


6 


5 


h 


3 


2 


1 




NOTE : 'The - 1 wo— va-P-i^nb-l-e s- 


-marked 


with 


an asterisk (*) 


we re 


used 


in th 



regression analysis shown in Table 6.2, where they are called "Initiate 
proiect for outside funds^' and "Initiate projf^ct if can be continued." 
The remaining variables were also used in a wide variety of preliminar2/ . 
analyses; these* variables were not significantly related to continuation, 
except for one variable — viz., the more important "state or federal agency 
suggestion" was to the initiation of the project', the less likely it was 
to be continued. 
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analysis make it clear that superintendents committed to a change 
effort can., strongly affect an innovation. 

In terms ,of data analysis, we could examine only one characteristic 
of the superintendent that seemed to matter consistently: the superin- 
tendences tenure. Table 6.2 indicates that. Title III projects were 
sonewhat^ (though not significantly) more likely to be continued if they 
were in districts where the superintendent had been in charge for a 
number of years. We attribute this weak effect to two tendencies in 
the. data. First, 13 percent of the sample projects received new super- 
intendents during the life of the projects, and the new superintendents 
seemed somewhat . more concerned with starting their own projects than 
cont inuing . those of their predecessors. Secondly, projects that were 
expanded had a disproportionate number of longer- tenure superintendents. 
Perhaps it is easier to .expand projects in stable situations. 

In summary, organizational or bureaucratic dynamics of a school 
system have pervasive effje-cts' on both project adoption and continuation. 
Whereas this hypothesis should come as no surprise, it does challenge 
an implicit assumption in the "seed money" approach to educational 
change — namely, that the educational merit of a project, demonstrated 
during a trial period, is the principal condi tion* of its continuation. 
This assumption is naive, because of internal LEA bureaucratic considera- 
tions and because of local political forces, which we turn to next. 

Lo*-:al Political Considerations 

The idiosyncrasies of local poli tical pressures on school districts 
make analysis of what happens to federally funded projects a very chancy 
affair. We asked a large number of. questions in both the Phase I and 
Phase II surveys about these political forces but, aside from one or two 
variables, we could not identify conpapable political factors that sig- . 

nificantly affected the continuation of Title III projects in bur 

12 ■ ' . . 

sample. However, school districts as public sector organizations are 

particularly vulnerable to issues of substance and politics pressed by 

their constituencies, and to fluctuation in the good will of their main 



The political effects on bilingual projects funded, by Title VII, 
ESEA, were more noticeable and more identifiable. See Vol. VI. 
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sburce of support, the taxpayers, and of their .vested supervisors, the 
school board. 

The quantitative data provide at least some indication of the sensi- 
tivity of school district decisions to political concerns about continua- 
tion. During the period of our data collection, school districts all 
around the country were concerned with the decline in student test ^ 
scores. Table 6.2 shows that the more strongly superintendents believe-^ 

tli^ir districts had dif f icult^ies with community reaction to test sv:ot 

13 

the more likely was project continuation. These instances of contl 
tion primarily involved ".remedial" individualization projects. The most 
interesting aspect of this finding, however, is* that many of these con- 
tinued projects weiv ^';t perceived as particularly successful. We wonder 
if district officials felt under pressure to demonstrate they were "doing 
something" about problems. 



13 , 

The super in tenden t ' s survey asked abou t s tuden t tes t s cores as 
one part of the following question: 

Here is a list of some souroeij of r.nnfliot or diffiaultij that have 
faced school administrators in the last few years. For each one , would 
ijou tell me whether it has been a major concern in this district j not a 
concern at all, or somewhere in between. 



Teacher demands 

Desegregation and/or 
racial conflict 

Reactions to student 
test scores 

Communi ty groups ' demands 

Reactions to the cost of 
the schools 

Divisive reactions to edu- 
cational issues such as fun- 
damental schooling, sex edij- 
1 ation, discipline policy, 

fCC. 



A Major 
Concern c 

29% 22% 18% 12% 
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The positive effect of school district size on continuation 
(Table 6.2) is another interesting finding that can be - interpreted in 
several ways. Our earlier reports (see Vol. II, App. C) offered the, 
hypothesis that larger school districts have more organizational and 
political ''slack." They can juggle their discretionary funds, and they 
can continue a project at "one end of the district" in relative isola- 
tion from other schools and communities/ In any event, school district 
size was positiveJ.y related to (a) wheth.er the project received district 
funds after the end of the Title III grant (.14), and (b) the extent to 
which community groups favored continuation of the project (.13). 

In summary, the district continuation decision did not follow an 
R&D model in which alternatives were assessed according^to their edu- 
cational value. Clearly "unsuccessful" projects were likely to be dis- 
continued, hut the status of continued projects depended on the central- 
ity of the innovation, on its relative cost (though not on the district* 
financial situation), and on organizational-political considerations, 
rather than simply on educational merit. 

SUSTAINING INNOVATIONS 

^ A district decision to continue part or all of a project is not, 
as we have suggested, the "end^ of the story," or even an accurate fore- 
cast of the project's fate after federal fundfng ends. All the projects 
in our Phase II sample were continued and most -^jiere judged as - highly 
"successful"; yet, after the close of their Title III grants, many still 
experienced difficulty in being sustained. 

For example, the Phase II superintendent's interview inquired about 
the 'former Title Hi projects* chances of being sustained . Because 



14 , 

The specific question and distribution of the 98 responses were: 

(Likelihood of Being Sustained)* For those schools currently using 
project activities or methods, how likely are they to sustain them? 

r 42% '26% 18% 14% . 1% 

J L ^ . I I I _L I 

7 6 5 U 1 9 ' 
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likely 
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these proj-ects had been through their three-year trial period, had 
been relatively successful, and had been continued, one might expect 
that virtually all- of them would be considered to be safe and stable. 
Yet less thrfn half of these continued projects were so rated by the 
superintendents. Mor^-^over, the correlatiQn between the project's con- 
jtinuation status and its likelihood of being sustained (according to 
the superintendent's assessment) was only :28. The district's decision 
to continue a project — whether expanded, reduced, or at the same level — 
did not in short, appear to guarantee its long-run survival. 

Why not? To explore that question, the Phase II survey asked 
superintendents to tell us f "In general, what difficulties do you 
think schools might encounter in sustaining project activities or 
methods?" As was to be expected, given the nature of our sample, none 
of the respondents cited disappointing project outcomes or perceived 
failure. Instead, most of thSeir answers fell into three broad cate- 
goriies representing different aspects of the internal operation of the 

school organization: . finances, personnel, and politics. Table 6.4~ - 

indicates the frequency with which these factors were mentioned. 



Financial Difficulties 

The most frequently mentioned district problem in sustaining a 
special project was financial — insufficient resources to carry on the 
project at the desired level of operation . "^^ However, the financial 



15 ' 

The correlation between the superintendents' perception of the 

projects' likelihood of being sustained and the superintendents' per- 
ception of the percent goals achieved by the projects was only .1*2. Thus, 
whereas the "failure" of a project predicted discontinuation, the dif- 
ferences among relatively "successful" projects may be unrelated to the 
problems they encounter xjfter continuation. 

Though more than half of the superintendents indicated financial 
problems with sustaining the innovation, these problems occurred for 
both more and ^1 ess expensive projects. Thus, the superintenden ts '"^ per- 
ception oft^the project's likelihood of being sustained- was only weakly 
and negativfely related to its funding level during the period of the 
Title III grant (-.12) or to its funding per student (-.09). Nor was 
the superintendents' perception of -the proj p^-^ s '. likelihood of- beirig 
sustained strongly related to adequacy of the districts' overall flnan- ' 
cial situation (the correlation was .16). These data are consistent 
with the interpretation e?tplored in . the text: Aside. from the need/ for 
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Table 6.4 

DiFFicULTif^^ In sustaining projects 

pROM S^^^^t^iNTENDENTS* VIEW 



Numl)L»r of 

F i n an cu a I • . 38 

Personnel^^ 

Stciff t:urnf>'^'*^*r* • . I 

Stc^ff rcsL.^^^in^'^* ,12 

AdiT n isL rn t ^ ^ ^'^Uk»rsli Lp .... S) 

Staff "burO^^^Jl-'' • . .^2 

Thtal pe^'.^un'^'^' I ' 33 

Pol it ica ] 12 

Dit'fiL'ultius ^^^rt' ^^nivl^tvab k» • IQ 

Other 3 



'^96 of loO re.^^^^^ndents aiiswored the question:. 

"I>i .ji:}u val^ i^nat'^ 'Hffu'uLtie^* do Ifou think :ujhoola 

mathodr>?" . • ' 

^Borauso .^t^t' ^'^'NpondiuUs ^;nve more* than one 
reason for pc* ^'^^o'^^^^' 1 d H' f i lmi 1 1 i^H , the subtotal 
of tlu' items ^tt^**^' OxfO^^'ds th(.' figurt^ of 33 givon 
in the t i^hi'-^^^n^ timn . 



dif f Ictilties perceived b/. th^^"^^ central office respondents took a number 
of different formn. A fe^ 0^ t\^e financial problems cited by superin- 
tendents resulted* from ^^^^ that the districts undertook Title III 
projcH: ts^_tJb^j_-Lhii-y^_cou.lci-^ot^ ^'^^'-^^o'^^^llJ afford to continue on their own 
•funds; exa-mples are proj^^t^ ^^^t relied heavily on aides, field trips, 
special alternative facilit^^^^ qV access to outsicle computer facilities. 

More common were financial difficulties arising from unanticipated 
or "hidden" operating co^^^'^. A ^umber of superin tendon ts ind icated that 
project operations were i" j^^^^^r^Y because the district had not planned 
for such technological ni^int^^^nce costs as those for repairing cassettes' 
that were central to a r^?di^^ program or for updating the materials in. 



school districts to have ^ g^^^^ally higher level of monetary support, 
the f in;incial^dif ficulti^^s e^^^Unf^red by change agent projects are 
primarily management and t>udS^t allocation questions. 
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scicnce kits tliat Formed the core of a Title III science instruction 
project. . * 

The most frequent source of financial problems, however, was the 
districts' failure to set aside funds for the support services neces- 
sary to sifstain the project. Many dis tricts seemed to assume that, staff 
development activit'ies purchased by federal funds were a "capital 
investment" which, like textbooks, had to be paid for only once (in 
•vsof t money) . this assumption failed to take into account either staff 
turnover or spreading of the project beyond its origlna^J. radre of vol- 
unteer tfs'Mhers. Moreover, in several instances, materials were de- 
veloped i) ne school and then '^1 isseminated to others without the ' 
trainiii - ipport activities that had made the original project work. 

In sliori ' J years after the end of federal funding, many superinten- 
dents and tM-nl.ral office staff reali^^ed that they needed to maintain or 
increase staLt development Ln order to sustain the innovation. .Btit 
bee iUH*^ districts had not taken steps to inccjrporate project staff 
tra: "ig requirements into the district's, regular in-service activities, 
or i"o provide for it in the dis trict budget in olher ways (through 
training of district resource personnel, for example) they felt unable 
to afford the central vehicle for sustaining the projec-t. 

A fourth cause of financial difficulties is not so much a factor 
identified by the superintendents, but rather a theme that ran through 
their responses to this open-ended question. That is, districts had ' 
trouble sustaining projects because they continued to vieV them as 
"special projects" after federal funding ended. • Contrary to the experi- • 
mental or developmental assumption's that underlie a federal "seed "money" 
strategy, districts generally^do not make provisions to incorporate the' 
proje':r once it has proven its merit. Even in our sample of relatively 
"successful" projects, . few districts appear to have rp.placp.rl_ previous, 
practices with the new practices. Instead of routinizing the chanj>^ 
agent strategies, districts instead tended to perpetuate the "special 
project" status. Doing so sometimes helped solve organizational apd 
poli t ical problems in the district — problems relating to equity between . 
schools or parental opposition, for example — but nonetheless, m.nde the v . 
project extremely vulnerable as budget deliberat ions proc':»eded.' *' 1 1 is 



much easier to trim funds from special* projects than from what the 
district represents. as its "education program operating budget" — 
an area still held somewhat sacrosanct by school board members, parents, 
and schoo\ administrators alike. 

In summary, though a few. of the financial difficulties associated 
with sustaining project operations were the result of events that could 
not have been anticipated by district administrators (such as the unex- 
pected withdrawal of the Catholic school partner from the^ project) , the 
vast majority of the problems were questions of budgetary allocations 
that could have hePTi predicted and could have been planned for. One im- 
plication that clearly emerged from the preponderance of essentially 
foreseeable financial difficulties is that, even within this, relatively 
select and special sample, di^^trict administrators of ten exhibited a 
budgetary management s£yle at cross-purposes with the assumptions funda- 
mental to a "seed" money" approach to educational^ change or reform. A 
"seed money" approach presumes that when projects are successful, the 
.district will then replace existing practices with project strategies 
or make thefei a permanent addition to the district's repertoire. Our 
data indicate, however, that such postfunding strategies were seldom 
followed; instead, projects tended to retain their special, status aiid 
their vulnerability to the financial fortunes of the district. 

Personnel Difficulties 

The second major difficulty involved district personnel. Like the 
financial' problems, the personnel, problems were varied but more or- less ^ 
predictable. .The ones most frequently cited related to-^ staff turnover. 
Either through transfer or retirement, project staff dispersed over time 
and diluted the enthusiasm and expertise that projects need. Some dis-, . 
tricts tried to deal with this problem by intervening in teacher-transif er 
practices, but quickly found that inflexible union guidelines made trans- 
fers for the sake of maintaining project activities difficult if nat im- 
possible. Furthermore, very few distric-t|5 had planned ahead to continue 

training in project methods or materials. ^ 

. \\ 

^ Staff Pf^sistnnrte was mentioned with almost th^ same frequency. d 
Particularly where the district attempted to spread or ^xpand project 
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operations beyond the original core of teachers, officials met resis- 
tance from, teachers new to the project. .Staff resistance was also a 

<? ■ 

frequently mentioned — though apparently Short-term — problem for project 

implementation during the period of federal funding. For the original 

project staff, however, training and other support servicers in addition 

to active •participation, in, proj ect development helped overcome or temper 

teachers* resistance. Yet district officials typically ' did not provide 

these same devices to teachers new to project operations after federal 

funding had ended. '^'^ 

Another majpr proMem was teadevohip. A number of superintendents 

.noted that the latk of principals' support made it c^ifficult to sustain 

18 " 

or spr e ad pro j e ct op erat iqns^. abse-Tice .of leadership .of support 

at th^ 'central off ice level was also seen as a problri^m. At the end of 

federal funding, project directors, whose salaries had typically been 

paid out of soft money, often wera promoted or reassigned, or left the 

district for 'a "better" position. Because districts generally either 

did not replace them or assigned their function to a middle-management 

administrator already burdened with other . responsibilities , the usual 

result was a vacuum of technical expertise, bureaucratic know-how, and 

concern for the /project. 

The personnel-' problems associated with sustaining project opeta- * 

tions, in short, generally appear to be the result of the district's 

failure to. provide th6 necessary support, particularly in the area of 

training and central office leadership, that proj ects ^ continue to rer 

■ , ■ . ■ ■ ' 'i^ 

quire after the end of federal funding. 

Polit leal Difficulties ' . 

With few exceptions, the dis.tric-t's political problems had to do 



Somewhat related to the issue of teachei; resistance is the fact 
that, as at least three superintendents noted, .the original project;, . 
staff were bored or "burnt 'out." The same support activities that 
could ameliorate resistance on the part of staff new to the project 
could also help to "recharge" the original staff.. . • 

18 J ' ' ' ■. •< 

. The correlation between principals* support for the projecf and 
the superintendent's perception of the project's likelihood of being 
sustained was a significant .28. 
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with school board or community acceptnnce of the project. Projects 

occasionally conflicted with new board initiatives — e.g., 'an open 

classroom project was seen as incompatible .wi^th school 'board interest 

in going "back to ba^^ics." Most political problems, however, stemmed 

from insufficient "public relations" -on the part of the central office. 

or project staff. Some projects did not have enough visibility in the 

district; the school board and the broader community felt they were not 

apprised of project activities and accomplishments or .of their impor- 

19 

tance compared with other pressing priorities. As a result', super- 
intendents feared that these projects would have "rough sledding" at 
budg&t review time when school boards might severely reduce or even 
eliminate special projects to accommodate pressures from vocal community 
groups. Without active community support or a general consensus on the 
centrality of the oroject, local political vicissitudes and uncertainties 
cast doubt on t:he long-run stability of these continued projects. 

t - . 

Strategic Prob lems -and ^"Proposed Solutions 

Thus, the problems encountered by district officials in sustaining 

project prac tiGes- f oll"with few~e'x*c"eptions into three broad categories, 

1 20 • 

finances, personnel, and politics. Of 96 respondents, only 10 regarded 
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The variables most strongly related to the' superin tenden t ' s per- 
ception of the likelihood of the project being sustained were the pro- 
ject's centrality and it;s visibility; the correlations were .32 and 39 , 
respectively. The specific question about visibility and , the distribu- 
tion of 96 responses were: 

(Visibility) How much visibility' did the projnnt achicDP. within' ' 
the district? 

29% 39% 21% 8% 1% 2% 0% 

_J_^ L_ \ I r ■ I ^ ■ 

7 • 6 5 4 . 3 2 1 

A great- . . - . " None 

deal ' 
• 20 ' 

The difficulties that did not fall into these categories were • 
idiosyncratic. 'For example, one district had difficulty continuing the 
project because of new state regulations concerning pupil/ teacher ratios. 
Another dlstric t found the project no longer appropriate because of 
prioV miscalculation of teacher demand. It initiated its project 
t-O train a cadre of paraprofessionals with the expectation that the 
teacher shortage would continue; but as federal funding ended, the dis- 
trict found Itself facing *a teacher surplus', and .barely able to pay the . 
teachers they already had, let alone fund paraprofessionai assistant's. .. 



continuation problems as manageable. This low figure is striking, 
seeing tliat most of the problems raised by central office officials 
appear predictable enough and amenable to solution through planning 
"and management . For that reason we thought Phase II would uncover an 
abundance of strategies designed .by .local districts to deal with conr- 
tinuation. Accordingly, we asked superintendents 'to tell us: "Do you* 
have any sug'^estions that would make if easier for other districts to 
sustain the activities of innovative projects similar to \h±s one?*' 
A total, of 80 of the 100 respondents replied. We expected them to 
draw on successful strategies, they had empldyed as well aS on, the wis- 
dom of hindsight. As we will* discuss, however, their suggestions were 
not useful., on the whole, and indeed implied th.^it few districts had 
developed" ..strategies fof change. 

'We believe this ^failure on the part of school districts ^ plan 
adequately for sustaining change reflects more general district^a^ti- 
tudes, two of which emerge from the superintendents' replies. The 
first and most prevalent attitude was. that the ultimate responsibility 
for sustaining changes does not rest with the local school district. ' 
From this perspective, school managers eit*her argued that the federal 

government should provide the, funds necessary for project maintenance 

<•■'''* 

and follow-on, or. expressed regret that such funds were not available. 
t' ^ . 

They made such comments as, to select a few, "What's needed is followr 
on funding, from any source; — federal government > state government, local 
taxes, foundations. The biggest problem with innovative projects is 
that the federal funding goes away and so the projects go away .... 
All this money comes in from Title III and then ceases ..^ What we need 
is continued follow- through , . . . A tapering offj of federal funds 
after the third year .. .' . or a longer period of funding . . . federal 
maintenance of successful projects .... The availability of seed 
funds and then continued . support of services from the state or USOE . . 
provide a procedui^e where the federal g'overnmerit could phase out some 
funding ah'dv then phase in some other funding so the first wouldn't be 
cut off suddenly." Such comments are direc tly contrary to the assump- 
tions underlying federal "seed money"; they can be seen as an unwilling- 
ness on the part of school- adminis trators to take responsibility for th( 

..." ' / 
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fiscal, fate of projects they initiated. In this context, the lack, of 
strate'gies for sustaining change is not surprising. 

The second district attitude' was more realistic and more rare. 
This attitude was that the responsibility for sustaining project changes 
rests squarely on the district. As one superintendent commented:* 

■ , The first thing [is that] a district should never 
apply [for Title III funds] unless it can sustain[the 
prpject] . itself ; at deast^f or thre^' ye'k after the ending 
of '. funding. Inability to implement the project after the 
grant is over is just throwing n;dney, away. Title. Ill pro- 
grams fail, because districts l^^tch onto the money for the 
..sole purpose of getting funds jand have no plan to sustain 
the proj ect . 

Rather thah faulting federal policies for their lack of follow-on 
funding or "abrupt" termination, school managers who took this atti- 
tude applied for and spent federal funds with special attention to 
.their short-term nature and acted to maximize longer— term local bene- 
fits from special funding mechanisms such as Title III. 

In our Phase II sample, as well as^ in our experience, districts 
subscribing to this second point of view were rare. However, there . 
was a subsample of about ten respondents, who did see the Title III 
funds as seed money and managed the funds accordingly. When these 
sup^r^intendents off:ered suggestions cpncerning what districts could 
do to sustain the changes associated with a Title III project, they 
generally agreed not only about what it was important to do, tut also 
about when it muist be done. In part?.cular, they emphasized the impor- 
tance of generating isupport for the proj^ect in the school -community 
and of the early incorporation of the supports necessary for making 
the project a' standard part of distrTct operations. In both of those 
areas, they/stressed that . activities aimed at project continuation 

must be planned and begun at the same time the project proposal is 

,1 

being developed and project implementation begins. The problem of 

sustaining a project, in other words, is not ah issiie to be considered 

' ' 21 
only when the last* federal check arrives. 
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y . The quantitative analysis also offers some suppdirt for t,He, im- 
por-tai\ce of planning for a project's continuation at its inception. The 
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The superintendents- advised that the '^support of the community and 
the. school staff should be cultivated from start to finish during the 
period of federal Support,, not mierely when continuation decisions are 
to be made and implemented. One superintendent commented: "The most 
basic thing to insure the continuation of the project' is to involve 
the teachers, students, and members of the community in the very be- 
jginning.." Similarly, another said: "You've gpt to convince your Board 
and your community at the outset that it is worthwhile; after you get 
into it , you have to work t-o keep the support 'level high. " A third 
superintendent ^explicl'tly acknowledged the necessity to conduct a 
""public relations" effort if support for continuation is to be had' 
when federal funds terminate: "During the grant period, generate a' ^ 
high, visib"* li-ty ..for successes- — particularly in the areas of student-^ 
growth and teacher acceptance. A project" needs high visibility- from ) 
s^he outset." 

Mobilization strategies that involve and inform.^the community in 
the project's early stages can give the district the political backing 
'it will need later on. in dealing- with the personnel and political prpb- 
lems that typically face a special pro/ject continued on district funds.. 
Asi. one superintendent-suggestied : "You have to have meetings to sell 
the program to the community- by presentations to ethnic groups, sports 
groups, all kinds of groups. Inform them.. You don' t^ sustain anything 
that is not supported by the community because that is where the taxes 
come from." To build isupport within the school board, another superin- 
tendent stressed: "You have to relate the project to accepted goals 
and objectives of the school board," , 

Aft-er federal f utiding .ends , effort^ td mobilize support for the 
project should be stepped up. Specifically, the teacher resistance or 
principal indifference encountered by many districts as they attempted 
to spread project operations could be ameliorated by providing the same 

orientation (or "sales") sessions plus involvement in planning for 

>■■•'. ■ fl 

o ■ — ^ — • : . — 

problem-solving variable, "Initiate projects if -the district can con- 
^tinue," had a strong and positive correlation (.31) with the superin- 
tendent's perception of the likelihood of the project being sustained. 
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^ implementation that were provided for the original project staff. A 
number of district managers mentioned the wisdom of assigning original 
project teachers to many of these advocacy or training roles; doing so 
not only excites the interest of other teachers, but also "recharges" 
the original project teachers. One superintendent advised:. "Just get 
it in . the budget and go out and do a. lot of selling to the teachers. 
We had. a lot of visitation — we found they wbulcf' learn from one anotKer." 
In ^hord, 'these superintendents foresee lukewarm support from the 
school board and resistance *from nonproject teachers unless . aative 

^ measures are taken from the outset to combat these continuation problems. 
The second strategic area — early planning for the Incorporation of 
the project- — is related to the need to generate ongoing support. The 
first general pi^ice of advice offiered by these school managers is that, 
administrators should assume from the beginning that th^ project will 
become a regular part of school operations — not simply a special project. 
As one superintendent said , ^ it is hecessajry to "plan the project ftom 
the start to be self-sustaining." 

What perspective and activitl.es does this type of planning require? 
it implies an ' integrated effort to ^ incorporate the project in all key., 
school district operations: the educational program, the budget process,' 
personnel procedures, and staff support activities. It assumes an in- 
tention to replace existing practices with special project practices if 
they prove, successful . A number of officials commented that an 
"incremental" view of federal projects such as Title III is bouncj to 
cripple tKeln from the -^s tart. One superintendent observed, "There is a 
need for new programs — but. they [officials in adopting districts] have 
to be willing to substitute new programs for old." Or^ as another • 
superintendent said, "You have to realize that once money comes 'in to 
develop' a* program, it can be carried on with local funds — because once 
it is started it is no mpjre costly than a regular program. 

Our respondents also suggested a number of strategies for incor- 
poration that explicitly address' the continuation difficulties discussed ^ 
'earlier. For example, like other respondents in our sample, this sub- ^ • 

' sample of school managers acknowledge the critical importance of training 
for district staff after federal funding ends. However, tRey also see « 
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a number of. ways thai the district can continue to provide this project 
support on their own. One str^egy relies on project, staff to function 
as trainers once the project is on. its own. "My main suggestion is that' 
there has 'to be a saturation relative to staff development. Tljie project 
should be able to sustain itself after the funding has ended and if you 
concentrate on staff training during the period ^of funding, these " 
teachers should be able to help you do this." Another superintendent 
suggested allociating special project funds to train fcentral office per- 
sonnel SO' that', as federal funds and the project director go away, 
someone is available to run training sessions and coordinate the imple- 
^mentation efforts in new school sites as well ^as keep the enthusiasm 
high in the original project sites. On this point, as one superintende.nt 
emphasized, "You have to train your regular staff to be able to handle 
what needs to be ^one and it has to be done in the regular work schedule 
of the school system." This latter comment emphasizes the ppint of view 
that is requisite to dealing with the problems of continuation — that is, 
that the project shbuld not continue as "special" but that the necessary 
supports s|iould be institutionalized into the regular school operation. 
Or as one superintendent commented, ". . .you have to fuse the project 
into the ongoing structures [and then] avoid words like * change* or 
'innovation;'" To accompli^sh this, "[District officials] should care- 
fully analyze the numbers of people dnd kinds of' activities • required to 
sustain operations during the /irs.t.year of research and development." 
Based on this analysis, then, provisions should be made within regular 
district budget and staff to accommodate these requirements. A central 
element, in this accommodation is to view project , funds as supporting 
installation costs, or to use the "soft" money to institutionalize the 
project. One manager cautioned: '^ake certain that during development, 
the project funding is spent on materials, methods and in-service 
training. This is necessary to do before funding is exhausted or frit- 
tered away on additional personnel — which are always lost aTter the 
project is completed." . 

In sumniary, the effective' strategies our respondents identified 
represent a particular view concerning the role. of special funding and 
an early, active involvement in planning for project support on district 
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resources. Thus the financial difficulties associated with continuation 
would be -seen as, budgetary allocation problems that would' be made 'more 
manageable by replacing existing practices with special project practices 
and • incorporating many of the support activities necessary to sustain 
whaX would be now seen as a "regiila'r'* school program. Personnel diffi- 
culties — resistance, turnover, "burn-out" — would bg addressed through' 
active promotional and training a.ctivities. The political problems that 
often hamper continuation would be meliorated through explicit attention 
to the building of a constituency for the project. - ,/ 

PATTERNS OF INSTITUT rONALIZATION . 

: — ■ " — t ' ... 

Thus far we have examined the continuation decision and the diffi- 
culties of sustaining continued projects from the viewpoint of the dis- ^ 
trict. It would be misleading, however, to assess continuation from 
that viewpoint alone, because in practice continuation .consists of what 
is done at the school level and, above all,, at the classroom level. What 
the district preaches and' what the school. practices may be two different 
matters. Sometimes it is even possible to, view a project, simultaneously, 
as continued because the district has* mandated continuation, and as dis- 
continued because schools and classrooms have ignored or paid little more 
than lip service to the district's continuation strategies and policies. 
Obviously, any meaningful analysis must consider both the harmony and the 
discord prevailing between the two levels. 'The analysis in the remainder 
of this section begins by reviewing the role of the continuation decision 
in the local process of change. 

The typical continuation decision was strikingly similar to the 
earlier decision to adopt a project. For one thiiig, the dominant act.ors 
were school district officials and school board members, rather than the 
district staff who were instrumentalo in* project implementation. Further- 
more, political and organizational . concerns again were likely to be dom- 
inant over the educational merits of the project and the educational needs 
of the district. For the purposes of , oUr\ analysis , however, the most 
important parallel was that neither decision was self-executing vlth 
respect to its content. Just as "adoption"\ did ,^ot. furnish an accurate 
forecast of implemented practicq-^ .neither-would a tally of "continuation 



d^cii^ions*' gjrve an accurate estimate of*. the return on the federal in-, 
vestment , 

Once adopted, change agent projects went through the difficult and , 
uncertain process of implementation, 'Volume III describes this process 
and delineates four patterns of implementation: technological learning, 

M - 

nonimplementation, caoptation, and mutual adaptation. Our Phase I 
research suggested that only those projects that followed a pattern' of 
mutual adaptation were ef fectively implemented and resulted in' signifi- 
cant change in teacher behavior. ' 

Similarly continued projects went through a process of institution-- 
alizationy which was as difficult and uncertain as implementation^ and' 
which likewise assumed a variety of patterns. Specifically, it. consisted 
of the interplky' ."between decisions' and behavior at the district level and 
at the classroom and school level; it involved both assimilation . of proj- 
ect precepts \>y school staff and incorporation of project procedures and 
activities by ^he , school system. 

. ' Our fieldwork- experience suggests, howeyer, that individual assimi- 
lation sometimes occurred even without formal district incorporation, 
and vice .versa. This "loose-coupling" produced f our ^af tei:ns of insti- 
tutionalization: . discontinuation, isolated continuant ion , x^^o' forma con- 
tinuation, and institutionalization. ^ 

Discontinuation occurred when neither district ' of ficials nor school 
staffs chose to continue proj ecf Operations in any form. At the district 
level, .this resulted either from an ^explicit decision to drop the proj- 
ect or from "benign neglect." tn either situation, a year after the' 
close rr^ the Title III grant, the' teaching staff were not using proj'ect • 
m^^^^ods or materials in about 70 of our original sample . of ,194 projects--* 
These total discontinuations occurred because the proi ects were not well 
implemented and therefore "failed." Most of them, too, seemed ancillary^ 
to the district's and staff's concerns in the first place; thiey were lie- . 
gun for opportunistic reasons and disappeared with the last federal check. 

Because neither district officials nor project staff seemed to learn 
much from discontinuation of this type,, the federal investment was essen^ 
t^ally waisted. Perhaps worse ^ project ^'failure""* sometimes bred cynicism 
among the project staff, not only'towa-rd federal policy arid district 
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of ficials but also toward educational reform, * Opportunism can leave a 

bitter aftertaste. ^ 

* . , ♦ * 

At least three discontinued projects, however, provided a signifi- 

cant learning experiertce in the view of project participants and dis- 
trlct officials. All tht*ee began a's serious efforts to meet educational 
•needs — e. g. , remedial math for underachievers — but their mobilization, 
activities ^did not ir)volve the staff nor engage their interest and 
commitment; ■ the)' were, J.n short, top-doWn innovations. One, a packaged 
curriculum innovation^ ess^entially broke down during implementation; 
the otfier two were altered to fit into the standard routines of the staff 
„and simply produced no real change. District of ficials wer<i aware of 
these problems, and told us that they had learned the importance of 
teacher participation and of mutual adaptation during implementation. 
Currently, these officials have begun to incorporate such strategies into 
reincarnations of the original Tit3^e III projects. ' ^ 

-. Learning from "failure" seemed to be the exception rather than the* 
rule, however,. It, appears to us that federal grant money generally did 
not serve an experimental, or tri,al-and-error , function in cases of dis- 
continuation. Organizational learning seemed more likely to occur on* 
continued 'projects. v • • ^ 

In the second pattern, isolated continuation^ district officials 
'did not ^actively or explicitly turn the project off, but gave it inade- 
quate or no support. The project methods or ipaterials were continued 
nonetheless — essentially in isolation from other school^— by project 
teachers who integrated them into their classroom practices. We observed 
a number of variations of this pattern. Fdt example, an elementary- 
school in an upper middle class community had 'a project involving dif- 
(f ■ ferentiated staffing and released time for teachers to do planning and 
training. Because district officials f^lt the ptoject could not work* 
in the more difficult conditions of other district schools, they gave it 
as little formal recognition and visibility as possible. The project 
was "tolerated" because of its popularity with teachers andr the comnunity. 
When federal funding ended, the principal tried to keep the project going 
on a greatly reduced budget and was denied a request to have a district : 
specialist assigned to the school to promote project strategies.^ In - f 

- : ^ . ^ . 
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another c<ise, a nationally validal:ed project involving multlage grouping 
was originated in a ''showcase" school. Although the school continued to 
receive a suppleraentary budget for' the ' "-spdcial project," district offl- 
cials otherwise ignored the project and gave only incidental support to 
other schools or teachers wanting to try project methods. These in- 
stances of isolated continuation were possible largely because of the 
"loosely coupled" nature of the school system. But without assured 
support from the district, such "pockets" of change "were likely to ex- 
perience^ the above-mentioned financial, personnel, and political diffi- 
culties of sustaining themselves. For example , they were vulnerable to 
staff turnover and principal .transfers . Moreover, Insofar as their 
operations depended' on special budget allocations or support activities, 

they remained subject to being severely hampered at the -first "financial 
22 

squeeze. " ' . 

In contrast to the""grass roots" . nature of' isolated continuation is 
the third pattern, which we call pro forma iJontinuation. , It represents 
a-situation in which the district establi'shed the innovation or some as- 
pect of it as official policy, but teachers did not- use the project very 
extensively in their classrooms. In some cases, school-level staff 
simply did not employ project precepts; in others, project methods or ^ 
materials were "continued" only in a ritualistic sense. For example, 
one reading project developed -specialized materials and' a curriculum 
keyed to diagnostic-pres.cr iptive procedures.. Because of political 
pressures (the school board wanted to combat declining test scores) and 
organizational considerations (dislrrict officials had. used part of their 
federal grant to "develop procedures foir markieting the materials and 
training manuals to. other school systems); dis-trict officials and the^ 
school board decided to adopt this approach to individualization on a 
districtwide basis. Site visits by the Rand staff confirmed the surveiy 
responses of teachers — thatds, teacher utilization of the reading 
^ . . • . :^ . ! 

22; ■ . . ' -^^ . • ' ; ' . 

In several msfances, superinffendents or assistant, superintendents 
chose a deliberate strategy of cutting Ijack on a project and keeping it 
in a special status to "buy time" until outside funding could be; found.' 
This strategy developed during the period of school .district growth and 
the inf lux .bf federal money. It seems particularly risky, in the 'current 
period of enrollment decline and financial retrenchment. 



program, was minimal and appeaifed to be only symbolic . Teachers con- 
formed with project forms and nomenclature but essentially ignored the 
substance project: strategies. Furthermore, district officials seemed 
aware of the* pro forma ritual Seing enacted- at the. classroom level, but 
apparently belieyed thdt the mere existence of the formal district man- 
date would help to mollify the concerns of school board members. . 

The final pattern of continuation^ institutionalization ^ occurs 
when project-related change becomes part of the standard educational 
repertoire at both the district and. the classroom level;., We'estimate 
^that between 5 and 15 percent of our sample of changie agent projects 
had been f ully institutionalized at - the „time of the Phase II research; 
or approximately two years after the end of federal funding. This 
figure is based on responses from the district-level questionnaires and 

.on fieldwork evidence for the 40 or so projects that had, according to 
teacher and principal survey" responses, a high degree of assimilation--*" 
i,e, , of continuation of proj ect "methods at the^^lassroom level. Most 
of, these relatively "successful" projects were still regarded as special 
projects and were still experienifing difficulties in sustaining them- 
selves. Only iabout ten had become regularized by the district so that 
they were no longer plagued by the financial, personnel, and political 
problems discussed earlier and "were no longer uncertain abou/.: their 
future ? . ^ . i 

The low proportion of "continued" projects in our sample that ap- 
pear to have been, institutionalized attests to both the difficulty and 
the complexity of . this process. In particular, because school districts 
are nof generally well- integrated organizations, in order for the con- 

^tinued project to be institutionalized , project support requirements 
must be^accommodated in most key -areas of district operations: the 
budget, personnel, curriculum support activities, the instrucuional ^ 
program, and facilitiles maintenance. Such accommodation of project 

The survey did not contain any relia&le measurements of these 
patterns and hence no quantitative analysis was conducted of them. Our 
impression is that most of the prpjects in the Phase II sample were of 
the Isol/ated continuation variety aiid perhapi^ one-third ^were p'ro forma. 
Even if these figures were reliable, they should not be taken as repre- 
sentative of all former Title III projects. 
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support requirements (e.g., the regular training necessary to project 
operations, the technological maintenance costs, centralized coordina- 
tion) does not simply "happen" as a result of a decision to continue. 
Rather, strategies of the type discussed earlier need to be developed 
for incorporating the project into district procedures; and this re- 
■quires the early and active attention of district officials to all 
relevant areas of system operations. 

To repeat, institutionalization is no more certriin or straight- 
forward than implementation. And our data suggest that, contrary to 
the hopes of federal planners in using* "seed money" strategies, full 
inrs titutionalization occurred in relatively few cases. 
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Chapter 7 
CONCLUSION 



This report has presented findings of Phase II of Rand's study of 
local innovative projects funded by federal change agent programs. 
Phase .1 identified factors affecting project initiation and implementa- 
tion. ^ Phase II explored the longer-run effects of federal policies 
aimed at stimulating local educational reform through the ;pVovision of . 
**seed vmoney/'; To that end, we examined a . sample, of ESEA Title III 
projects one. to two years after, the end of federals funding. This chap- 
ter summarizes the fitidings of. the study, which addresses rhree questions 

o What vdoes the contiriuation of specially funded projects mean and 

how should this process be assessed? 
o What influences the nature and extent of continuation, at the 

classroom level? * ' ' 

o How do districts deal with change agent projects at the end of.. 

federal funding and how do their actions affect the long-term 

fate of "the projects? V 

■ - . . ■ " " . ■ ^ ■ 

■One note of caution:. This was an exploratory study. - The quantita- 
tive analysis is subject to methodological reservations, and bur" quali- 
tative conclusions are open 'to' rival interpretations. These caveats 
notwithstanding, we believe the findings. ^summarized below provide a 
number of valid working hypotheses about the long-term prospects for 
federally sponsored educational change. 

'THE MEANING OF CONTINUATION 

A central theme of our research is that "continuation'* is a com- 
plex phenomenon that cannot be accurately assessed merely by tallying 
district decisions to continue or drop projects. Such an approach would 
be misle'ading for a number of reasons. For one, few projects'" in our 
sample were continued precisely as they were implemented during their 
period of special funding. District officials chose among a variety of 



options: Some decided to expand project operations, others to reduce 
or eliminate some project components, still others to reallocate person- 
nel and streamline procedures, and so on. Similarly, depending ,:.on what 
they learned from the project, teachers often modified their classroom 
activities in idiosyncratic ways as they continued usiiig various aspects 
of project methods or materials. Consequently, evexiy.^^n the level or 
scope of a project remained approximately the same, "the substance of 
project-related activities was typically modified after funding ended,' 
to reflect the preferences and' priorities of individual staff as well 
as the fiscal realitieB of the district. ' 

Because the "continue;d" components of a project may (differ ; signifi- 
cantly from the shape they took when the innovation was originally 
adopted, it is best to think of continuation not in tCirms of a formal 
py.:bject>strutture but in terras of the persistence of project-related 
changes after feder " ^'unding ends: Continuation should be defined in 
terms of the continuing effect of "specially funded activities on class- 
room practices. 

It is no easy task, however, to assess the continuing effect of an 
innovation. One difficulty that confounds attempts to apply aggregate 
measures to project outcomes is the "loose cdcipling** of school district 
activities: A decision at one level in the system may dr may not have 
a significant influence, on behavior at another level. For example, a ' 
district may announce its official decision^ to continue a project, but 
the extent to which teachers continue to use project methods and materi- 
als may be only incidentajLly related to that decision. Conversely, the 
district may drop a project, but classroom teachers may elect to conr 
tinue some of its features on their own without formal district sanction, 
or even knowledge. Similarly, a . central:j)f f ice decision to continue 
project operations at sele^cted schools may be effectively meaningless 
if. teachers respond with mere pro forma compliance. Ah- aissessment of 
the extent of continuation therefore must encompass the decisions and 
actions of both the* district and the classroom teachers. ^ 

At the classroom level, the crux of the matter is the extent to which 
teachers have assimilated project methods or materials into their, regular 
classroom, practice. At the district level, the issu& is the district's 



commitment to the long-term stabTlity of . tlie project. This commitment 
requires the ,inaorpoy\ition of project requirements into such district 
operations as budget, personnel, curricuiTim, and facilities planning. 

In the end, however, effective continuation depends on the choices 
and behavior of • classroom teachers. Unless they have assimilated projr 
"ect method^ or materials . into their .classroom activities, continuation 
will^ to no more than ritual. But..if they are to receive the 

support they need to sustain proj ect-rrlated changes in the. long run, 
the district git its level must incorporate the necessary arrangements. 

. FA CTORS INFLUENCING PROJEC'r CONTINUATION^: IN THE. CLASSROOM ""' ^ 

The complex interplay between proj ect characteristics and local 
institutional setting ^determined how well^ projects were, implemented 
duriqg the period of federal^ funding, and how much change they produced 
in teacjie^rs' behavior. .We found that effective implementation was 
-■essential -to the teachers* assimilation of project metHods and materials 
and to tlie contJ.i3jLied use. of the projects in the classroom. Therefore, 
we sought to identify project arid- institutional .factors that ^ affected 
continuation both directly and indirectly. ... ° 

In general, neither funding levels nor educational methods employed 
had a significant effect either on initial project outcomes or on con- 
tinuation'^in the classroom^ Project "success*' — both short-term and . 
long-term — had little to do either with the .size of the federal grant 
or with the particular educational technology pursued. Rather, the 
likelihood that the federal, investment would "make a difference," and 
the usefulness of a particular educational technology , were largely 
determinecf by local choices and local institutional characteristics. 
In other words, ,the project's' re^^ources and educational methods mattered 
less ,than how it was carried out. ' 0 

Local Choices " ^ v ^ . 

Two kinds of local choices had major effects on continuation in 
the classroom: " the scope of change attempted by the project and the 
implementation strategies selected to put project plans into practice. 

The scope of change attempted — its amount, its complexity, and the 
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.effort it required of project staff — is a project design factor that 
can vary even for similar educational technologies. It strongly 
affected, project continuation' for at . least two reasons. First, teachers 
responded to challenge: The more change that was asked of them in their 
classroom activities the more change they were lively to make. Second, 
teachers were likely to take ambitious projects seriously and to see 
them as an opportunity to improve their skills and grow professionally; '" 
but they needed, to have the goals made clear to -them during implementa- 
tion. When this was done, they, responded with "the commitment, enthusi- 
asm, and extra effort required for effective implementation and change. 
Not surprisingly, when teachers worked h^rd to carry out an ambitious 
.change effort, they were likely to , continue using- project methods of 
materials after the end of federal funding. \ - 

'•The implementation.- strategies chosen* for a project constituted a 
second critical factor. Two strategies were particularly important for 
promoting the learning and motivation of classroom teachers : their 

participation, in project decis^ions, and staff s^upp.ort activities. 
' ■ ' /* ■ ■ . * 

Teachers^ sharing in decisionmaking about the project's adaptation 

gave them^a sense of "ownership," aided- implementation, and, thus en- 
hanceid th^ likelihood that they would continue using what they assimi- 
lated from the 'project. Their participation was facilitated by staff 
meetings and especially by local materials development. 

A well-executed staff support strategy proved crucial to ef fec'tive'' 
implementation and continuation. A number of valuable :project activities 
can accotlipany training: assistance from project or district staff-, ref'gu- 
lar meetings^, the use of outside consultants, observation of other class- 
rooms, and the like. When those activities . were practical and timely 
in ^the ij^ojects we studied, they were a major determinant of whether 
skill-specific training given to prpject teachers actually led to r long- 
term change in classroom jpractice. Taken together, staff support ac- 
tivities and teacher participation in project decisions promoted mutual 
adaptation — i. e'; teachers could adapt the change agent project -to the 
reality of their own classrooms, and in turn be changed by it.. These 
strategies thus helped teachers to assimilate and continue new practices. 

210 
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Institutional Factors ^ 

Two institutional factors Influenced project continuation in the • 
classroom: the quality of management leadership and support, and the 
characteristics of project teachers. . , . 

TJie leadership of the project director and the school principal 
had a major influence on project implementation and eventual continua- 
tion. ..The project director, not surprisingly, was important to ef- ^\ 
fective implementation. A skillful director was able to facilitate 
the task-learning necessary to implementation,, and to specify clearly 
the goals and "methods of the project. Well-conducted staff, training 
and timely and practical staff support activitj(,es u'sually reflected 
the substantive expertise and skilled interpersonal relations of a 
good project director. " , . - 

Project directors', influence on continuation^however , was likely 
to be indirect, because they often were reassigned, wete promoted, or : 
left the district at the end of federal funding. ■■ ' \ ^ 

. The principal was the key to both" implementation and continuation. 
During project, implementation, principals lent moral and organizational 
. suppoVt, to the project director and . the project staff , ran interference, 
with nonproject staff and anxious parents, and created the organiza- 
tional climate of. acceptance for the project in the school. Without. 

ii • ^ • ■ • - * ■ ■ 

that encouragement, staff typically did not expend the. extra time and 

effort necessary to effective implementation. After the end of federal 
funding, the . principal influenced -continuation in Mually direct ways. 
Often, because of turnover in the original cadre of project teachers, 
:pr6jects would have decayed without active efforts by principals to 
bring on :new st-'aff. . ' . , 

- Principals often sent subtle but. nonetheless influential messages 
concerning the legitimacy of project operations in the school— messages 
'that teachers could not help but receive and. interpret in terms of 
^their own professional self-interest. It was extremely difficult for 
teachers to go on using project methods or materials without the 
principal's explicit support . -Finally, because the districts in our. 
sample often took a laissez-faire attitude toward ..the project after the 
conclusion of the federal grant, it was often up to the rprlncipal to • 
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"flght" for the project and protect it from the financial; personnel, 
and political problems that could erode its continuation. In short, 
the principal was the "gatekeeper, of change." 

The other major element of the institutional setting that influ- ■ 
ehced project continuation in . the classroom was the aharacteHetias of 
. project teaahevs. More experienced teachers seemed to be Tess flexible 
during implementation and less susceptible to change. Teachers who had , 
a sense of efficacy—who felt they could "get through to even the most 
difficult or unmotivated students"— were associated with more effectively 
implemented projects and with more project-related teacher chringe. The 
verbal ability of 'teachers had no significant effect on implementation, 
teacher change, or continuation, but was positively related .to improve'd 
student performance. • ' ' • 

These .result^-: raise questiong alsout the design of change agent ' • 
proj,ects: Is it possible to instill a new willingness to .'change in 
vet^jan teachers? If not, our findings imply, that innovative projects 
should be staffed w^th efficacious, less "resigned" .teachers .. That >. 
raises^ a further question: Is it ' p6ssible- to enhance , teachers ' ■ sense 
of efficacy? Districts-can always handpick staf& for pilot projects , 
of course, but that, amounts to a delaying s-trategy if the eventual in^ 
tent is to spread innovations or to maintain them among the general run 
of teachers after the original cadre of teachers move on to other tasks. 
If teacher characteristics can be Biodif ied , ■ then projects could be 
staffed with the 'usual m,ix of personnel ,. ^and appropriate staff develop-' 
ment strategies could be employed to compensate for staff shortcomings. 
This strategy would enhance long-run continuation if its staff develop- 
ment component were also equipped to handle personnel turnoveir. Though 
our quantitative data do. not address, these issues,, our field . experience 
suggests that staff development activities could be-iised.to raise the . 
sense of efficacy and to rekindle the enthusiasm of most teachers. 

Continuation of project methods and materials, in summary, does - 
not depend to any significant extent on the level of.project funding 
or^ on; the particular technology undertaken by the local project. It 
ultimately depends on the motivation of teachers ; . principals , and d is- 
trict personnel^ and on the choices tl^ey make to implement the project 
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and change their behavior. Continuation at the classroom' level depends 
on how ambitious the" project was to begin with, the implementation 
strategies selected to carry out project plans (part icularly teacher 
participation and staff support sactivities) , on the institutional 
support for the project, and on characteristics" of the staff . 

HOW DISTRICTS DEALT WITH PROJECTS AT THE EW O F" FEDERAL FUND.INC; 

^ The end of the federal Title III grant confronted school districts *' 
with a decision that many did not seriously plan for when they originally 
adopted a project: They had to decide to support the project in whole 
or in part with district resources, or not to support it at all. The 
latter choice—to discontinue the .project—wa^ often easy because the 
project had not been effectively implemented. This implementation 
failure, as well^ as subsequentc distontinuatioh, typically reflected a . 
district attitude of opportunism toward the , project . ' Ordinarily, dis- 
continued projects .had been started ta take advantage of available 
federal funds rather than to deal with real needs,, were not central to 
Ifhe :distric.t' s major educational objee;tives, and received little insti- 
tutional support"^ dur.ing implementation. 

The decision, to continue a project was, in contrast, more diffi- 
cult and ce^pttplex, for it required some degree of , district financial, o 
organizational, and* political commitment. Fl^hancially , the end of 
federal funding usually resulted in sharply reduced project budgets, 
particularly for expensive projects. Innovations that had used federal 
money to lower the student-to-adult ^riatio (e..g. , by hiring aides) were 
"ctftHack to live.with'^in .the district budget; projects that had. spread 
their resources thinly across both elementary and secondary grades were 
severely reduced in .their scope. In most cases, districts" did not pro- 
vide supplementary funding for. ''continued'* projects. . - 

This downward adjustment r.gor continued' projects/'did riot depend on 
the district*' s .overall f inancial situation as much as it did on organi- 
zational considerations :\ For example', the central office and staff 
X _ , • " ■ ■ V. .■ 

k't^titude^unde^lying project adoption tended to persist to the time of 

continuation. * Opportunistic projects '"were treated with benign ^neglect, ... 
if they^v/ere not discontinued; projects beguq with an explicit commitment 
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from central staff were often continued, and sometimes e^xpanded, even ■ 
though they had been no more "successful" during the period of lederal. 
funding than other projects. The original organizational commitment 
to the project usually declined when key personnel left the district 
or were reassigned. Loss of leadership hurt the status of the project 
most severely when ' a'^;superintendent who had championed the proje^ct left 
the district. But- transfers of principals, loss of project directors, 

and reassignments of central office staff also eroded institutional 

- ■ ' ■ . ' ' ' ; 

support for continued projects. This erosion often occu'^rred at a time 

when support was most needed to deal with project staff turnover , and 

especially with local political 'problems . 

The basic political difficulty faring continued projects arose 

f.rom the need for school board approval of a new project status. . School 

boards often passed on th6 original proposal for a federally funded. 

"trial" project without serious debate, becaus^. it did not seem. to 

compete for district »re^oirf c'e*s7'''''At* /ft end of the. federal grant, how- . 

t * • 

ever, the project' lost, its "special projection. " It became subject to 
close scrutiny, and requests for financial, personnel, and . ^facilities 
support had to be weighed against other claims on district resources; 
to survive, in short, i't required a new legitimacy in the system's 
political arena. Perhaps because these innovations were usually small- 
scale,_ central office staff rarely prep'a'red the school board or the 
community for the project's full acceptance.. In any event, most con-- 
tinued projects retained their "special" status rather than being in- 
corporated into the. district ' s re^lar educational repet;tpire. Two 
years after the end of the federal grant, they still were vulnerable^ 
to financial, organizational, and politT^al instabilities. 

Thes^ continued projects appeared to fall into two general patterns. 
The first and most prevalent pattern %n our sample can he^called isolated 
continuation. In this instance, district administrators adopted a 
laissez-faire attitude that left project continuation to the discretion 
of' school level staff.' The continuation of project-related changes, 
then depended on the extent to which project staff had assimilated 
project precepts during implementation and choise to integrate them into 
their classroom practicies. But without the active support of district 
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.officials, such continuation was .bound to be sporadic — typically con- 
fined to the original cadre of project s\^aff — and thus their persistence 
was threatened by staff turnover, principal transfers, financial pres- 
sures, and-t"he like. ..... \ 

A second, pattern can, be" called pro forrriq continuation » /In. this 
situation, district officials explicitly decided to continue the proj- 
ect, but despite this decision, teachers did not use project-related 
activities extensively in their classrooms. This essentially ritualistic 
pattern of continuation came about for several reasons. Some continua- 
tion decisions were themselves symbolic--f or example., a central^ staff 
decision made primarily to mollify school board concerns. 

Beyond a; formal announcement of the intent to continue a project, 
district officials did little or nothing to support the continued opera- 
tion of praject activities in the schools. Pi'O' fovma cotiitinuation also 
resulted from inappropriate or ineffective district-level continuation 
strategies. If - district officials simply "mandated" the installation 
pf project activities in new sites, without first eliciting the support 
and commitment j^f staff new to the project, both .teacher resistance and 
inadequate teacher preparation often made?/ project operations only super- 
ficial. Or, when district administrators failed to. provide the support - 
necessary to maintain project-related change — gaining the commitment of 
principals or maintaining required levels of material aid, for example- 
teachers yere not able ..to continue the project fully. 

The majority of continued prpjects, in summary represented either- 
isolated or pro fcrma continuation. One' other pattern occurred' in our 
sample, however., in which proj ecfc-related change became ^^integrated into 
regular operations at both the district and classroom levels — that is, . 
the projects were institutionalized in whole or in part. They shed 
their "special" status and rep'Zaced .practices that existed before the 
project began. Although this pattern occurred ihf requentj]^ , its char- 
acteristics .suggest strategies that local policymakers. might follow to 
secure the long-telrm l^enefits of a change agent project. " ■. 

Institutionalized projects planned for eventual continuation from 
the outset — when the project proposal was . developed. The*- central office 
staff always aimed to replace some existing practices with the project. 



but were wise enough not to "oversell" the project's merits in its 
early phases. Though the scope of the change effort typically was 
limited to receptive schools, district officials paid early attention 
to mobilizing broad-b^sed support for the innovatioi^. And after fede- 
ral funding ended, mobilization efforts were increased in order to 
pave the way for the project's transition from its "special" status 
to its incorporation into key areas of district operations: the 
budget, personnel assignment,' curriculum support activities, and the 
instructional program. 

In the' budget area, the project changed its stat-us from a sp^ial 
line item to an activity absorbed in the opei;atlng budget. This change 
was possible becjause school board members, who had to vote ^or inclusion 
of the project into the regular operating budget, were kept informed 
about the project and its ^accDmplisTiments.-. In the personnel* area, re- 
placements for key project members -..were allowed a period of "learning 
on the job" and project participants were used as "training cadre." 
In the area of curriculum support services ,. some districts had incor- 
porated project staff training requirements into the district's regular 
in-service activities, and others .had trained district resource person- 
nel in project-related skills using the federal grant money. Finally, 
the replacement of existing practices in the instructional area meant' 
that the support of principals - and teachers had to be enlisted so that 
the project would work at the classroom 'level, / . 

The basic problem iif achieving effecitive continuation, then, is 
for .district officials to avoid the trap jof^Viewing institutionalization 
as automatic; on the contrary, it emails idr 'J;remobilization" and ^ 
"reimplementatioh. " They must cotne. to realize that the groundwork and 
planning for sustaining a change agent- proj ect requires the early, - - 
active, and continued attention of school district managers. 

CONCLUSION \ ■ ■ \ ■ ' 

■ o Having reviewed the federal "seed money" approach , to promoting 

education reform, our. overall assessment of its contribution to date 

■ ■ ' ' , ' . ? ■ , ' 

is mostly- negative. Federal funds have stimulated the local adoption 

■ . . . . ' . . I . 

of a wide^variety of innovations, but adoption does not assure effective 



impleijentation . Nor does effective implfenvehtation guarantee the loitg- 
run survival of proj ec t-telated improvements. . 

This ♦report has argued that the effective continuation of a change 
agent project — its institutionalization — largel/y depends on whether 
teachers assimilate project strategies and integrate them an to their 
classroom practices during implementation. Hpwever important federal 
money is in getting the project started, neither its availability nor 
' Its amount heavily influences , classroom practice. Nor doesi federal 
funding strongly affect those factors that really matter in promoting 
effective implementiation , teacher change, and classroom continuation: 
local de'sign clioices, local implementation strategies, staff tup t i vat ion , . 
and school climate and leadership*. , . " 

Institutionalization of a project also depends on the nature of 
the continuation decision " reached by district officials, and the strat- 
egies selected *l:o implement .it. District, of.ficials typically do not 
make this decision'solely on the basis of ""the project * s educational or 
technological 'merit, as' a *'seed money" model assumes they Will. Instead, 
.other., organizational and political factors moderate* and determine the 
Continuation status"^ of a change agent project, even if the project has 
demonstrated its'value. What is most discouraging, few districts in 
our sample approached the end p£: fedefral funding with ins titu tionaliza- . 
'tion in' mind . Instead, their 'budget and personnel decisions perpetrated 
the "special project" status of innovations and left them vulnerable to 
the cif inancial ..fortunes of the district. When a distr ict decides to con- 
tinue a project permanently, that decision i^ not self-executing : 
Jt}^:it* jir.s for cpntinuatiori have to be developed. If staff are not jffo 
resist "mandate'ci" new practice^ for example, district officials .^ce 
again have to win their commitment and support. Unfortunately^, very 
few districts appear to have the management' capacity to develop and i'm- . • 
plemerit-^ continuation strategies that effectively sustain . the changes . 
resulting from successful innovations. . . 

Nonetheless^ total pessimism is not warranted. Our . negative 'as- 
sessment, is based 'on the low f re que nay of significant and institution- 
alized change — but such change did. occur, though rarely."' A primary 
purpose of Rand*s study has been to learn from both kinds of experiehce,' 
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SO that we can identify factors that promote or impede the institu- 
tionalization o'f successful innovations. We 'believe that understanding 
these factors can provide a realistic basis for the formation of more 
effective local, state, and fedep^l policy aimed at educational reform. 
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■ .. ■ Appendix A' 

STATISTICAL TAiiLES FOR CLASSROOM-LEVEL ANALYSIS 



This kppendix dispiays statistics for the classroom-level arralysis 
as presented in Tables ^5..1, 5.2, and 5-3. Table A.l below, lists- 'the 
means anci standard deviations of the independent and dependent variables 
used in Tables 5-1 and 5.3 (n = 499)v Table A. .2 shows the. zero-order 
, correlations bfetween the variables (plus the additional varia^lW, ^ . 
Difficulty of Implementation).- Table A. 3 presents the/partial corre-. . 
latiens af regressing each of the (exogenous and endogenous) vaTiabies 
of Table ,5.3 on all the other varia^bles;, these partial correlations 
provide an insight into the extent of multicollinearity in . the data. ' 
Generall;/ speaking, for these data, a correlation of .07 is signifi- 
cantly different from zeto at ,th.e .01 level.^ Tabl-p A'.*'>^ displays the ; 
(uns tandardized) regression coefficients and standard errors for the 
regress ipns^ of Table 5.1. 
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. * KEY TO VARIABLES : TABLE A. 1 

. * ■ ' • ' » 

(1) Percent goals achieved 

"(2) Total teacher change, 

(3) Funding level of project Clog) 

(4) Funding per student 

(5) Number of project schools . 

(6) Co.mmunity involvement . 

(7) Individualization techniques 

(8) Use of specialists 

(9) Classroom organization change 

(10) Curriculum revisions ' 

(11) Specificity of goals ' / 

(12) ,. Extra effort, required of staff 

(13) Overall change required in teaching 

(14) :,. Staff paid far training . 

(15) .Training received (usefulness) 
(16,) Classroom- assistance (usefulness) 

(17) Consultants (usefulness) . • . 

(18) Project meetings (usefulness) 

(19) Observation of oth.er classrooms (usefulness) 

(20) ■ Partiodpation in project decisions 

(21) Quality of working' relations 
(22") Principal- support ' 

(23) Project director effectiveness- 

(24) Elementary school - * , 
^(25) School size , 

(26) School socioeconomic setting . 

(27) Teachers' sense of efficacy 
(2.8)- Teachers' experience (years)^ 

(29) Teachers' verbal ability . . 

(30) Difficulty of implementation , , 

(31) total student change 

(32) ' Continuation of project methods 
j;33) Continuation .^f project materials 

•f , ■• ■ ' • ■ . 
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Table A.l ' 



MEANS AND STANDARJ) DEVIATIONS OF CLASSROOM VARPABLES 



Variable 






Standard 






Number 




Mean 


Deviation 


Minimum 


•'Maximum 

*■ lO A 111 U 111 


\(1) 


68 


.058 


24.736 


0.0 


100.0000 


(2) 


19 


.118 


5.8583 


4^0000 ^ 


28.0000 


(3) 


ir 


.395 . 


0.70'365 


9.2252 ^ 


12. 7220 , 


(4) • 


167 


.73 


257.68 


11.1230 


:2695.6000 


(5) 


5 


.5760 


5.5566 


1.0000 


44.0000 


(6) 


0 


.41020D-02. 


1.0341 


-1.-9340 


2.4400 


(7) 


0 


.•98032D-01 


0.97934 


-2 .6140 


2.1210 


(8) 


0 


.11012 


1.0032 


-2 .0410 


3.1470 


(9) 


0 


.17182D-01 


1.0^36 


' -2.0400 


3.1570 


(10) 


0 


.12023 


, " 1.0096 


^ -2.3120 ' 


3.0.950 


(11) 


5 


.3520 


• 1-3^71 J 


i.pooo 


.7.0000-. 


. (12) . 


5, 


.8040 


1.433tr 


i.booo 


7.0000 


... (13) 


0, 


.29200 


0.45468 


0.0 


1.0000 


■ (14; 


0, 


.60400 


0.48906 


0.0 .. 


1.0000 


(15) 


4, 


.6286 


1.7553 


I.pooo 


7 .0000 


(16) 


. 4,0520o.\ 


.2.2193 ' 


1.0000 


7.0000 


(17) 


3. 


.3680 


2.4472 


1.0000 


7 .0000 


(18) 


4. 


.3760 


2.1332 


i!oooo . 


7.0000 


. (19) 


1. 


,8020 


2.5610 


1.0000 


7 .0000 


" (20) 


4. 


0380 


1.9911 


1.0000 . 


7.0000 


• , ».'(21) 


- 3. 


9380 


1.0743 


1.0000 


5.0000 




7:4060 


1 .^8 3 3 3 


1-0000 


9.0000 


(23) 


'■ 5. 


4900 


1.6155 


1.0000 


.7.0000 


(24) 


0. 


72400 


^ 0.44702 


0.0 


1.0000 


(25) .. 


' 610. 


01 


359.97 


350 .0000 • 


2500 .0000 


■ (26) 


2. 


2820 - 


0.90470 


1.0000 


• 5.0000 


(27) 


9. 


7380 ' 


1.4756 


5.0000 


12.0000 


. (28) 


12. 


102 . 


7.9631 


1.0000 


40.0000 


(29) 


42. 


460 


5.6061 


. 11.0000 


- 50.0000 


(30) 


4. 


4780 ■ 


1-7520 


1.0000 


7.0000 


(31) 


. 13. 


092 


2^:7913 


2 , 00.00 . 


18.0000 


(32).'. 


.5. 


1820 


1.5326 


1.0000 


7.0000 


(33) 


.5.. 


8060 ■ 


0.91453* 


4.0000 


7.0000 


• NOTE': ■■ 


For key 


to variat le 


numbers, see 


preceding page. 
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Table A. 2 

ZERO-ORDER CORRELATIONS BETWEEN CLASSROOM VAR 



V.ir labK- 












































( 1 ) 


(2 ) 


( 3) 




(j) 


■(h) 


(7) 


(K) 


(9) 


(10) 


(11) 




\\ 3)" 




(15) 


(16)* 


( 17) 


(IK) 


(19) 


(20) 


( 1 ) 


I .01) 




















^ ■ 












.1. 








(- )* 


. jj 


1 .00 






































(J) 


.01 


■ ov^ 


I . 00 




































(i) 






. jH 


1 .00 


































( j ) 


. tlO 


-.12 


. 1 7 


- .21 . 


" 1 .00 
































(n) 


-.0(J 


. 1'^ 


.OK 


.O''. 


-.LO 


Koo 






























(?) 


. 1 3 


. Ih 


.2'j 


.21 


-. J K 


-.Ob 


1 .00 




























(8) 


.OS 


-.00 


..10. 


.00 


-.Oh . 


-.09 


-.0> 


I .00 


























(y) 


.07 


AH 


.Oh 


.02 


-. 1 1 


.04 


-.01 


.01 


1 .on 














• 










(10) 


..rJO 


' ..)7 


- . 00 


-.04 


.04 


-.04 


-.04 


.03 


-.01. 


I .00 






















(11) 




.2b 


. 12 


.11 


.02 


.05 


.28 


.Oh 


-.01 


. 1> 


1 .00 




















(IJi 


■'.17 


. 32 


.oa 


.02 


-.15 


.20 


.2:1 


-.00 


. 10 


.13 


.25 


I .00 


















(1 J) 


.0(1 




".03 


.00 


-. 10 ■ 


. 1 b 


.01 


-.09 


.12 


ill 


• 04. 


.15 


1 . 00 
















(U) 


• -.01 


.Uri 


.02 


-.02 


-.14 


.04 


. 10 ■ 


.05 ■ 


■ -,04 


. 10 


. 10 


.^14 


.09 


1 ;oo 














(1 :0 




. 31 


.00. 


.OJ 


-.10 


. 14 


-.20 


.OK 


.OS 


.'11 


'■ .4 3 


■ 


. 10 


.15- 


I .00 












(lb) 


. 3H 


.27 


-.0 3 


.11" 


. - .04 


.15 


.02. 


. 14 


' .07 


.12. 


.30 


..13 


.02 


. -.01 


. .44 


1 .00 










(17j . 


. J4 


.2H 


.09 


.03 




.29 


.13 


.03 


.09 


..02 


.19 ■ 


.12 




.lb 


.41 


.29 


1 .00 








■ (IS) 


.'Jti 


*.2y 


.Of) 


.04 


-.13 


.19 


■ .oV 


.05 


. 15 


.1; 


.26 ■ 


.28 


■ .10 


.16 


.45 


.35 


.37 


■ 1 .00 






(19) 


.09 


.14 


■ . 12 


.04 


-.07 


.20 


. 10 


-.09 


.24 


-.00 


.11 


• 12. 


.10 


-.00 


.24' 


.22 


.23 


.25 


1 . 00 




(JO) 






.00 


.02 


-. W 


.21 


.Ob 


.08 


.17 


.11 


.22 


.26 


^.14 


.16 


.28 


^ .24 


.26 


.4 5 


. 19 


1 .00 


(-'D 




.23 


-.03 


.Ob 


T.'05 


^02 


.12 


.02 


.07 


.IK 


.39 


.27- 


-.02 


.17 


. 34 


.23 


.IK 


.35 


.,13 


.32 




. . . 


.Ifi 


.00 


..00 


-.00 


.09 


.01 


" .05 


.05 


.04 


.27 


.12 


.00 


-.05 


.28 




.15 


.17 


■ .11 


".1-2 


. (23) 


.31 


.20 


-.02 


■ .13 


-.05 


. 12 


' .02 


.09.. 


.04 


.13 


.40 


, .11 


■-.01» 


_ .02 


.38 


.44 


.21 


.35 


.14 ' 


.24 




.li 


.03 


-.07 


.04 


.08 


-.24. 


. 12 


; if 


.00 ■ 


.01 


. 1 1 


-.01 


-.19 


-.17 


.01 


.11 


-.02 


-.OJ 


-.00 


-.Ih 


(23> 


.Oi 


.01 ■ 


.09 


-.07 


.0 3 


.05 . 


-.02 


-.13 


.00 


.03 


-.03 


. 10 


.10 


.15 


-.03 


11 


-.02 


.07 


-.09 


.13 


(2h) 


.03 




-. 10 


-.20 


.09 


.Oi 


-.•01 


i.07 


.00 


-.08 


.02" 


.06 


-.00 


• QJ 


.01 ■ 


-.07 


-.00 


-.07 


.00 


.Otr 


(27) 


. '2 


.15 


'.I'j 


.02 


-.05 


-.03 


.01 


-.02 


.00 *■ 


.02 


.14 


.08 


-.01 


-.05 


.lb, 


.12 


.09 


.13 


.01 


.15 


(2S) 


-.07 


-.OH 


.09 


-.01 


.01 


.04* 


.01 


-.01 


-.09 . 


10 


.OS 


.03 


-.00 


.02 


.Oh 


-.04 


. 10 


-.01 


.08 


.0.3 


ciy) 


-.00 


.02 


-.04 


-.04 


-.02 


■-.0.1 


.08 


.02 


.00 


■ .09 


,.07 


.08 


.00 


.05 


.00 


-.00 


.00 


.05 


-:oo 


.03 


(.io; 


-.26 


.Oi 


.07 


..04 


-.06 


.11 


.06 


-.11 


■ > 1 2 


.07 - 


-.11 


.21 


:1 1 


.04 


-.10 


-.12 


-.02 


■. .00 


-.01 


.04 



Kfv to Vririablc numbers: , - ' . " 

t * 

{\) Pt-Tccnt- >5onl.s achi^vt'd (11) Spirclf Iclcy of gonl-s (21) Qunlltx^ftf working* rcliitlor 

(2) To^al tirachtT ch.Tnf;f (12) Excrii effort required of oCTff " * (22) Prlnclp^il support 

(3) Funding level of project (lo^) (13) OveVall changcv. required In tearhlnr ' , (23). Project director cffectlver 
. (4) funding- per student (14) St.tff paid .for training (24) Elementary schqol 

(3) Number of project^ schools „ (15) Training received (usefulness)' (25) School size ' - . 

(b) Coraraunlty Involvement v..(l^) Classroom iisaist.ince. (usefulness) <26) School soclooconomic settlr 

(7) -.Individualization techniques (17) Consultants (usefulness) , ^ (27) Teachers* sense of efflcncv 

(8) Use of speci.i 1 Ists (18) Project meet loRS (useful nessj (28) Tenchers* experience (yearw 

(9) Classroom lirKaniza t ion change (19) Observation of other classrooms (usbiulncss) (29) Tenchers* verbal ability 
(10) Curriculum revisions (20) Participation In project decisions . .,(30) Difficulty of inp leniontat Ic 
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Table A. 3 



partial' CORRECTIONS BETWEEN CLASSROOM VARIABLES 



fiumb^T .(1) (2) (3) (4) (ft) (?) 



(8) W (lO) i(ll) (U) (13) (W (15) (16) (!?)■ (18) (19) ,(20) (21) (22)''(23) (2^, 



(I) 
(2) 
(3) 
li) 

■ (7) 

(9) 
dp) 
(11) 
(I.') 
(13) 
(K) 
(13): 
(16) 
(W) 
(18) 
tl« 
(iU) 

(12) 
(23) 
t2i) 

(2?)' 

c^) 



.51 

.14 .11 

.02 ,05 

,0] .01 

.04 '..03 

-.10 '^0 

. -02 , .02 

(.-.01 -.00 

.01 .08 

.03 -.03 

.lb .02 

-.02 .17 

-.02 .16 
-.IM .03 

.11 M 

.09 M 

' .08 ,.08 

-'.0? .03 

-.05 ,04 
.09- ■-.02 

.21 .01 

.10 sOl 

' .23 -.03 

.10 ..06' 

.10 .01" 

" .05 -.08 ' 

.12 ..M 

-.10 -.13 

-.03 .04 



(25) (26) 



(2?) (2H) (29) 



.26 

:i9 .18 • 

.28 -.2r .24 
.04 .114 -.05 

..14 ..12 -^.21 - 

.04 ..Oj -.05 - 

.09 -.00 -.10 - 

.04 -.02 .03 - 
.10' .01 .10' 

.02 ;.04 -.0? 

.00 -.00 .-.05 

.01 ,7.04 -'.ll . 

.06 -.06 .".02 - 
.05 .05. .04 

.04 -.03 .00- 

.t)7 -.03 -.03 

.07 -.02 -.00' 

.or -.01 -.10 ^ 

.11 .05 .02 - 

.01 -.06 -.02 ' 
.06 .11 -.02 . 
.05 .01 J .,08 

.05 -.06" ,07 -,0' 

.08 -.16 .09 
>15 .05 .-.02 

,12 -.00 '-.00 

07 -.04 



JO 
-.08 
--.09 . 
-.11 

'J8 
.16 
-.02 
.04 
.'II 



".03 
.05 
,00 
.00 

-.0.7. 

.;-.06 
.13 
.03 
.02 ■ 

-.04 

-.07^ 



.11 

.03 
'\00' 

,03 

.02 
-.09. 
, .06 

.03 

.08 ■, 

.02 
-.00 
-.12 

.10 
-.06 

.03'* 

..03 

,.04 

-.09' 

-.07 

-.07 



00 -.00 .08 .03 



.14 

-.06 
.04 



-.02 
•'.01 
.02 
;03 
.19 
.10 
.02 
.00 
'.00 
.03 

.or 

.03 
.03 
'.If 
.02 



.14 . ' • 

.03 .40 . 

.09 M .27 

.12 .04 .05 
.07 .05 
-.01 .17 
.03 .03 
'.02, -.05 

.05 -.05 .09 

■.01 -.02 -.01 

.00 ■ .03 M 

.07 .1^3 .11 

■v03 .03 .01 

il6 -.06 

.02 .05 



.15 . 
■02 .0.3 ,17 
04 .02 .06 
00 
11 



.04 
.0] 



.00 -.05 .09 



.06 
.01 
.12 



.01 :06 
.06 .00 
.02 .07 



.09 .04 .02 .-.I 



Kfv u» vjridblf numbors: 



-.02 


-.02 


.20 


.37 




















.06. 


.10 


.17 


.06 


.31. 


















-.00 


.04 


.20 


, .06 


.15 


.42 
















.02 


".05 


,:07 
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Table A.^- ' 

I 

REGRESSION COEFFICIENTS AND STANDARD ERRORS FOR CLASSROOM 



IVrii'iit I'rclw'i 
fio.'ils Ai'lili'Vt'J • 



Toliil 
TimcIkt Qm^y 



lii'pi'mli'nt V,iri;ihli' 
Tiil;il 

Student iHipriivini'nt 



CiinliiiiiJiliiiv iif 
I'roliHl Mi'tlinds 



lwlt'|inndi'nt Viirl.ihh' 

I !1 tadlnu h'Vi'l o[ pn'ji'ct 

{^) (•'tiiidini! per studont 
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(nl CuBininitv Inwlviwnt ^ 

. U) lndlvid(ialii!iitlon ti'clini.qui's 
i:sc of -ipt'oUllsts 
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'(H) 'Staff paid for' training 
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(17) Cansiiltani:!: 
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.(311) Participation In project decisions 
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(.']) Pniject dlri'ftpr effertiveness 

(2i), Fleiientary school 

US) SclllHll si?x ■ 

(2h) School siu'loi'connmlc setting 

(2?) Teachers' sense of efficacy 

(28) Teachers' experience (years) 
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Kegresslnil 
roefflclent 

LOOTS, 
0.25T52E-02 

•i.diW 

•Q.27969 

0. 53't29 

1. ^336 
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Error 
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Krror Coefficient 
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Error Coefficient Error 
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^ Append Ix B 

THK SOURCES OF VARIA TION IN TEACHER RESPONSES ^ 

A continuing theme of' this research has been the extent to which 
the oijtjL'omes of projects depend on the yay they are carried out — that 
is, on their imp lementatioti . We have argued that the strategies whereby* 
ci project Is Installed and modified to meet local needs may.be much 
more critical for results than is the educational theory underlying an 
innovation, or the. initial project des^ign . As a consequence, we would 
expert that teachers in' the same "innovative project- — and even in the 
same school — would have, dif ferent . experiences with thq innovation, 
diipendiruj on theiv imvlementation experience q. . In other words, we 
would'say that an i/nnovative project is not a uniform educational 
treatment; rather, it is a loosely structured intervention in the on- 
going life of a school, and its- effects depend on the often unique ways 
that it is actually carried out in each classroom. 

Because we expect the results- of a project^to be affected by its 
implementation in individual classrooms, we have based our analysis 
riot only on data collected for whole projects or schools, but also on 
data gathered from more than one thousand teachers. Each teacher was 
asked to report on the innovative project in his orher classroom. In- 
the two .surveys we conduc ted . (in 1973 and 1976), we found that the< 
teachers varied considerably in such area^ ais 

o Their understanding of the project; . ^ ' 

o The extent to which they took part in designing, preparing " 

for, modifying, and adapting the innovation; and ^' 
o The^ extent^ to which the innovation influenced their teach- 
ing practices. 

Moreover, our 1975 survey revealed that project teachers, principals/, 
and district officials often sharply disagreed about the outcomes of *' 
a^ project. (We argued -that this disagreement between teacher«s -and 
their superiors may have been due to systematic differences in the 
way implementation affected them.) '\ ' 
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Both our data and our theoretical understanding of the chaoge 
process lead .us to. believe that teacher-to-teacTier variation' is an" 
important part of the story of every .innovation • But how important 

• are these differences empirically, compared with the differences 
among projects and among schools? It was possible that the variati(^n ■ 
among teachers in 'our sample was an artifact -of the .way in wh:^ch we 
collected the data; perhaps, that is, it merely reflected dif*ferences * 
among schools and projects. If this were true, some of our analysis 
might be invalid, particularly our conclusions about relationships 

. involving nonschool ^nd nonproject variable^. This appendix explores 
this issue by assessing how much of the variance in teachers' answers 
is due to teachers being in the same school or on the same project. 
The analysis supports the basic assumption made by Chaps, h and 5. 

ANALYZ-ING DIFFERENCES IN^ TEACHER BEHAVIOR - * 

-We need to- estimate the relative iftagnitudes,, of several' sets of • 
behavior: behavior that varies between individual teachers (thf'.r 
idiosyncratic behavipr) , behavior that varies between, innovative proj- 
ects— (represent ing: the uni-f-ied~responses-of-~teacherS^^ 

and beha'Crior that varies between project teachers in particular schools 
(representing the effect of policies in different proj ect settings) . 

By use of the . s tatis tical technique known as analysis of covari- 
ance, we may break down the parts of teachers '/ responses- in the follow- 
ing partition : ' 

Teachers' responses = (part due 'to the., pro j ect ) plus (l) 
leftover nonproject component. i- 

: ^ ■ ' > 

From such a* breakdown, we can measure the proportion of variation in / 
teachers' responses that. may be attributed to the pro j ect . . For .example , 
if the. classroom-to-classroom variance in a project's success were very 
small, then factors common to all classrooms and teachers in ithe project 
wepe more important than differences among teachers. Once we have esti- 
mated the amount of variance among' all the sampled teachers taken to- 
getner that is due to each, teacher's particular project, we ean compare 

229 
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it with the amount of variance that would be statistically attributable 
to random groupings of teachers Mot necessarily.c in the same project. 

« If the difference is large, we know that the project per se "deter- 
mined" teacher responses; if the di fference. is small , then merely 
knowing what project a- teacher belonged to ^tells us little about tJ-'^ 
teacher's answers. ' ' 

A second comparison can be made between the effect o'f the project 
on teachers, and. the effect of a particular school within a project ■ .> 

!, on teachers. This is done by comparing two statistical formulations: 

i • Teacliers' responses = (part due to project) plus *' (1) 

leftover nonproject component.- 

Teacherj^ V responses = (part due to school within the (2) 

the project) . plus leftover noris'chool component. . » . 

Because, all schools in our survey took part in some project, Eq. (2) 
cannot account for less of the variance in teachers' behavior than 
Eq. (1)7"^ If the school formulation (2) accounts, for substantially 
more of the variance in teachers' responses than does the project 
formulation (1) » then knowing at which school a teacher taught within 
a project adds significantly to the precis'ion of an estimate of that ^ 
teacher's response, even beyond knowing which project the teacher took 
part in . 

In summary, by use. of a statistical analysis of covar iance , ..we 

' •> ' 1 . ' - 

'make two' comparisons : , 

• .-■■{ ^ - ■ . ... 

o First, we compare the variance explained. by the project in 
which each teacher worked with the variance explained by 
purely random groupings of .teachers; 

o Second, we compare the variance explained by the project in " 
•which each teacher worked with the variance explained Vy 

c 

— 1 ' - :" 9 ■ ' ■ 

^ The comparisons are based on the sta.tis tic, the amount of vari- 

ance "ekplained by" the independent variables. The independent variables 
consist of a .set of dichotomalis v>aria{)les standing for .each of the proj- 
ects and each of the schools, Vhere teachers in our sample were interviewed!.' 

2S0 
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• the part icular' school within n pruject to whi(Mi n tonchov 
Wc'is Mss Lgnod . , 

From these two comparisons we derive estimates ol' the maj^nitude of 
pvojecu. effcctiii and school effects on the hehavior of, iucHvidual class- 
room teachers. " . / 

THE VARIANCE IN INDEPENDENT- VARIABLES 

Tables B.l and B.2 show the results of the statistical tt^sts of 
whether teachers* behavior is individually id iosync rati , is shared by- 
teachers in the same school, or is shared among the teachers in a proj- 
ect. Separate statistical analyses were performed for iweiiLy v^jriabl.es, 
which are defined and analyzed in Chaps. A and 5. Table B.l shows the 
extent to which teachers* responses covaried ,iCcording t(i their project. 
The rS t ronges t effect of teachers' innovative projects on their responses 
was found for the group labelled "Very high." Wlien there was no dis- 
cernible correspondence of teachers'' responses to the i r i iinova t ivc 

• ■ r 

projects, variables were labelled "Insignificant" (because the pattern 
of responses was not significantly different from those we might expect 
without knowirig each teacher's innovative project). Levels project 
effect on/teachers between these j:wo extremes have been categoriezed 
"Moderate," or "High," according to the value of the "increment due to 
project." Table B.2 shows the extent to which teachersl, responses co- 
. varieid according to their ^ school , beyond the coVariance that wa^ attrib- 
utable to their project. When the effect of being in a particular school 
W4S most strongly reflected in teachers' responses, the indicator is 
categorized "Very high" in school ef fee t ; When .^a teacher's school had 
no apparent effect on the pattern of responses, beyond" the effect al- 
ready me'asured for the innovative project, the indicator is categor- 
ized as "Insignificant." Table B.3 summarizes all of these findings 
regarding the strength o.f project and school effects on teachers' 
reports . . 

Insignificance of Bac!:>^round Related to School oi; Proj ect 
"< . ■ • 

Teachers" reports of their background characteristics showed no 
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Tabie B.l 



TEACHER VARIANCE EXPLAINED BY TEACHER'S PROJECT: 
INDEPENDENT VARIABLES 



Variance 


< Increment 




. Explained 


Due to 


> • • 


in Test 


Project 


Significance 


ANOVA (R2) 


(Test R2)a 


of Increment 


.241 


None 


Insigaif leant 


.238 


None '. . 


• Insignificant 


.235 


None 


Insignificant 


.258 


None 


Ins ignif ieant 


.353 


.095 


Moderate 


.270 


' .012 


Moderate 


. 3^9 


. ' .061 • 


Moderate 


.267 


.009 


Moderate . 


.346 • 


.088 • 


Moderate 


•• .283 


. .025 


Moderate 


• ■ . 283 


^ .0125 


Moderate 


.351 


.093 


Moderate 


.-299 


.041 


Moderate 


.358 


„.100 


Moderate 


.393 


.137 


High 


.365 


.107 ■ 


High 


.386 


.128 


High 


.515 


.2.57 


Very hjlgh 


.590 


■' .332 


Very high - 


. .472 


.2-14 


Very high 



Variable (Teacher's Responses) 

■ ■ • - n 
Experience^ (years) 
. Sense of efficacy in teaching 
.Verbal ability 

Usefulness of first-year training 
Specificity 5f goals 
Overall change required 
Project directo,r effectiveness 
Usefulness of classroom assistance 
Frequency of staff interactions 
Usefulness of pretrajLning . ' 
Quality of working relacions 
Principal support ( 
Usefulness of later training 
Use of specialists 
Usefulness of consultants , 
Extra effort required 
Curriculum revisions 
Individualization techniques 
Coramurii ty invo lvement_^ 
Classroom organization^ change 

NOTE: Entries are based on a regression "of each variable on a vector 
of dichotomous .variables, each ^corresponding to an identifier of a proj- 
ect in the survey. R^^ from thiis basic 'regression is shown in tHe setond 
column. " . . s 

^''Increment Due to Project'* ts the* difference between the variance 
explained in the test . regression (R^'f and the R^ that could occur by 
chance if the independent variables - (here , the identifiers of particular 
projects) were not related to the dependent variable. The; statistical 
test for the significance of this relationship is F(67, 353) ^ 1.36 for 
p < "0.05. This corresponds to a critical value of the R^ statistic for- 
a" ^'signif icant*\regression of R^ ^ 0.259.' 
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Table B.2 



TEACHER VARIANCE EXPLAINED BY TEACHER'S SCHOOL: 
INDEPENDENT VAR:;ABLES 



Variable (Teacher's Responses) . 

Sense of ^efficacy in tear.'.iing 
Verbal ability 
Experience -(years) 
Usefulness of classroom assistance 
Overall change required 
Usefulness of first-yejjr traini'-.g 
Project director ef ^^ectlveness 
Frequency of staff ..interactions 
Specificity of goals 
Individualization tc "hniques 
Usefulness of pretraining 
Extra effort required 
Usefulness of consultants 
Quality of worScing relations 
Curricu3.um revisions 
Community involvement 
Usefulness of later training 
Use of specialists 
Principr^.1 support 
Classroom organization change 



Viiriance 
Explained 
-in Test 
aNOVA (r2) 

.353 ' 
.36^* • 
.383 
.401 
- .404 
.408 
.450 
.477 
.483 
^.07 
:438 
.499 
.519 
.446 
.524 
.684 
.471 
.519 
^554 
.601 ' 



Critical 
Value 
for 
Test r2 

.395 
.393 
.398 
.418 
.421 
.,411 
.^460 
.481 
.487 
.615 
.431 
.496 
.518 
.431 ' 
,513 
.675 . 
.443 
.490 
.485 
.581 



Increment 
, Due to 
School Significance 
(Test R2)a of increment 
ft » 
None Insignificant 

None Insignificant 

None Insignificant 

None Insignificant 

None Insignificant 

None Insignificant 

None Insignificant 

None Insignl^f icant 

None Insignificant 

None Insignificant 

.007 Moderate 

.003 Moderate 

.001 Moderate.. . 

.015 High 

.011 High 

.009 High 

.028 Very high 

.029 Very high 

.069 Very high 
.020 



Very high 

NOTE: Entries are based on. a regression of f?ach variable on a vector of. dichot- 
omous Variables, each corresponding to fin identifier of a school in the survey. r2 
from this basic regression is shown in the j econd column ^ 

^"Increment Due to Sphool" is the differetice between the variance explained in 
the test regression (r2) and the R^ that could. occur by chance if the independent 
variables had no explanatory power beyond that of the project regressions (reported 
in Table B.l). Since the increment required to establish stat.lstical sig?iif icaiice 
will vary depending on the variance explained by the project re^^ressions , the appro- 
priate, critical value' is reported in the third column. This value of corresponds- 
to the statistical test of whether 'F(56; 298) -^s 1.-38 for p < O.Oiw 



evidence that similar teachers tended to teach in partictiiar schools 
or projects. Their years of experience, sense of efficacy in teach- 
ing (their' avowed ability to ^"get through" to even the most: diff icuJ t 
or unmotivated students), and their performance on a brief test of 
verbal ability were not significantly related to the project or £he 
school ill which they^ taught. ° In other words, there was no tendency ' 
in oui? sample for certain teachers to be /ielected-'ro participate in 
specific programs (or schools within programs )v-or voluntarily to 
"select . themselves" into programs. Therefore, differences among 
schools or programs are probably not 'due to systematic differences 
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Table B.3 



SUMMARY OF VARIANCE, EXPLAINED IN INDEPENDENT VARIABLES 



Project Effect 



School Effect 


Insignificant 


Moderate 


Hlgli 


1 ^ Very High ' 


Insignificant 


Experience 
Sense of efficacy 
Verbal ability 
Usefulness of first- 
year training 


Specificity of goals' 
Overall change required 
Project director 

'effectiveness 
Usefulness of classroom 

assistance 
Frequency of' staff 

interactions 




■ 

Individualization 
te;chnlques 


Hade rate 




Usefulness of 
pre training 


Usefulness of consultants 
Extra effort required 




High 


I * 


Quality of working 
relations 


Curriculum revisions. ' 


ConsDunity involvement 


Very high 




Principal support 
Usefulness of later 

training 
Use of consultants 




Classroom organization 
■ change 



in the backgrounds of participating teachers. Surprisingly ,* teachers 
assessments of the usefulness of training provided during the project^s 
first' year also showed no effect from projects or schools. Although 
teachers experiences with training for an innovative project may well 
be very important to project implementation and outcomes, such experi- 
ences evidently vary widely, within schools and programs. Perhaps 
teachers' evaluations of first-year training depended on particular 
conditions they encountered in their own classrooms. 

.10 ■ * ■ . . 

Weak Association of Specificiry of^Goals and Overall Change ^ ' 
Required with Project 

Moderate effects due to their projects (but no significant, pattern 
of school effects) occurred for reports of two major factors: the spe- 
cificity of the projects' goals and the overall change required ±n 
teaching. While we would certainly expect a pattern of similar reports 
.^f or particular proj^ecirs" and not for participating schools on these vari- 
ables, the smallness- of the project effect is noteworthy. It suggests 
that teachers in the same projecL often differ substantially in their 
perceptions of project scope and goals. ■ ' 

; - In addition; teachers' responses moderately reflected their proj- 
. ects. (and not their schools), for three variables: project director 
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effectiveness, the -.usefulness of classroom assistance, and. the frequency 
of staff interactions. Teachers evi^iently were fairly diverse in their 
reactions to both project directors and classroom assistance, perhaps 
because of the importance of indlvidual.personality interactions in 
these relationships. 

■ T ■ . 

0 '. ; • • 

Moderate Effect of Proj ect and School on Preproject Training 

Teachers on the same project 'tended to evaluate their pre-imple- 
menration training, similarly, within projects, but t'here were measurable 
differences in how teachers, responded to the training depending on tUe - 
school they were in.' District policies regarding such training were, 
not washed out by individualistic responses ;. in addition, it appears 
that differences :between teachers' (and the problems they face) in dif- 
ferent schools may have caused the training to be evaluated differently 
in different schools. Perhaps because teachers; had no direct personal 
experience. .with the project^ when they received this training, their 
responses Were not completely fragmented cr idiosyncratic. Still,' the 



amount of teacher-to-teacher variation in 
high. " 



their assessments is quite 



Project Effects Due to Extra Effort Required and Use of Consultants 

, The variation in the usefulness of outside consultants, and in 
extra effort required reflected first the proje.cts, and then, to a more 
nwderate but significant extent, the . schools within projects. Because • 
outside consultants are generally hired by district officials, and such , 
consultants often give the same prepared advice and remarks to different? 
audiences, it is not surprising that a high. level of similiarlty in 
project teachers' reports was found. The school effect shows that, how- 
ever predictable, a,, consultant, might be, he or she is received differently 
in, different school settings. The importance of the. school setting 
certainly extends tO; problems it faces—and therefore, to the way its 
staff responds to a consultant's comments. The finding that teachers ' 
in a project* were in considerable agreement on the extra effort it 
required of them may reflect the project^s implementation more than its 
technology, or-design, or^students. Effort depends on implementation 



decisions regarding training, district support, and teachers freedom 
to adapt the project ro their needs. In addition, the sehool differ- 
ences in effort' suggest that the original design of the- project does 
not by itself account for the predictability of teachers', effort; 

Greater Effect.. of School Than of Project on Working Relationships 

Whire teachers' extra effort was strongly, related to the project 
served by each teacher and only moderately to the particular school 
within a project, the quality of working relationships depended only 
moderately on the pi;oject and. quite highly on the individual school. 
Because teachers e'xposed to- the same project and 'district policies • 
nonetheless display major differences in their views of working rela- 
tionships at tfie school level, we may infer that the organizational 
climate of each school influences the receptivity of teachers to new 
policies and projects. The statistical finding provides evidence for 
the assumption made in Chap. 5 that the uniquie identity of the school 
environment 'extends to the climate that teachers create for their work, 
quite independently of the effects of the district or project in-- which 
they work, , • , 

Variation of Principals' Support by School 

Teachers at the same school tend to view their' principal 's support 
for the project similarly, as Chap. 5 assumed. Teachers'" reports of 
their principal's support toward the innovation also reflected the 
teachers' district and project. The pattern of principals' attitudes 
is not unaffected by some of the overarching policies or character-. , 
istics of the project, across the individual schools in the project. 

Dominance of School Effects Over. Later Training 

Unlike teachers' reports regarding training they received before 
or during the initial year of an innovative project, their assessments 
'of training in later periods were, very highly related to the school in ^ 
which, they taught.- (There was also. a moderate relationship to the in- 
novative project as a whole.) This suggests that some of the classroom- 
to-classroom variability .that is associated with the diversity of 



teachers' initial reactions to the implementation of an innovation 

may be somewhat resolved by the passjage of time. In addition, it may 

be that school policymakers learn how to sharpen their training programs 

and make them more coherent as they gain experience with the project. 

Variability in Educational Method ' . . 

The use of individualization techniques was very highly related 
to the innovative project in which a teacher participated, and not at 
-all tp, a teacher's school within the project. Teachers understood and 
reported, with remarkable consistency , the extent to which a project 
used individualized methods. Evidently, project officials, can success- 
fully comm.unicate this- dimension of an innovation to teachers. More- 
over, there is essentially no systematic deviation from pro ject-wide ^ 
individualization reports by teachers in particular schools . 

Each of the other techniques, reflected substrant ial schopl effects, 
indicating, th^t an innovative project "mutatesV^ depending on the school 
in which it is tried. Community involvement showed a very high rela- 
tionship to the project as a whole, but also a high dependence on. the 
school within a project in. which it was employed. The particular in- 
gredients of cdmmunity involvement were evidently • affected by local 
conditions, and a common., project policy apparently left room for schools 
to dilffer in their use of commiinity-oriented techniques. 

The use of new classroom organization pa'tterns depended very highly, 
on each project but the school' also defined the innovatioiii,. These 
school-to-school differences show the difficulty of imposing such tech- 
niques as open education; or differentiated staffing on diverse groups- 
of teachers. They also show how miich these innovations tended to 
develop organically, in concert with the conditions of support . and 
learning that are present in each school.^ , . ^ 

Both project and school determined the teachers' descriptions of 
curriculum revisions. The variance in teachers' descriptions of "'the • 
use of specialists depended mostly on the school and moderately on the 
project;' 
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VARIANCE IN PROJECT OUTCOMES 

^ l^ables B'.4. and B. 5 give the results of the statistical tests of 
whether the variance in the dep^endent variables^used in Ghap . 5 re- 
flected the project, the school, or. individual variation among the 
teachers. Table . B. 6 suininari7;es all' of these findings regarding" the 
•Strength of project and' school effects. 

Changes" in Teaching . > 

Most 'of the behavioral outcomes, especially those focusing on 
teaching changes- that resulted from the innovation, bear only a 
moderate relationship to the overall projet.t (and* none to the teachers 
school within the project). The "pattern of teacher responses* regard- 
ing their basic skills ..teaching, affective development teaching, and 

Table B.4 

TEACHER variance" EXPLAINED BY TEACHER'S PROJECT: 



DEPENDENT 


VARIABLES 








Variance 


Increment 






Explained 


Due to 






in Test 


Project 


Significance 


Variable (Teacher's Responses) 


ANOVA (r2) 


(Test R2)a 


of Increment 


Change in basic skills teaching- 


. 306 


.048 


Moderate 


Change in affective development 


.300 


.042 


. Moderate 


Change in deal;Lng with special 








learning problems • 


.315 


.057 


Moderate 


Improvement in student. behavior 


.294 


.036 


Moderate 


Continuation -of project methods 


.279 


.021 


Moderate 


Change in classroom organization 


.414 


.156 


High 


Improvement in student achievement- 


.379 


.121 


High 


Percent project goals achieved 


..398 


.140 


High 



NOTE: Entries are based on a regression of each variable on a vector 
of dichotomou?^ variables, each corresponding to an. identifier of a proj- 
ect in the su^-vey. R^ from this basic- regression is shown in the second 
column. ' ' ■• 



^"Increment Due to Project" is 'the difference between the variance 
explained in the test regression (r2) and the R^ that could occur by ' 
chance if the. independent vari^ables (here, the identifiers of particular 
projects) were not related to the dependent variable. The statistical 
test for the significance of this relationship is F(67, 353) ^ 1.36 for 
p < 0.05. This>corresponds to a critical value of the' R- statistic for 
a "significant" regression of r2 ^0.259, " 
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Tabie B.5 



TEAC^iER VARIANCE, EXPLAINED BY TEACHER'S SCHOOL: 
DEPENDENT VARIABLES 



Variance 


Critical 


Increment 




Explained 


Value 


Due to 




In Test 


for.. 


School 


Significance 


ANQVA (r2) 


Test r2 


(Test R2)a, 


of* Increment 


.427^ 


.449 


None 


insignificant 


.373 


.444 


None 


Inslgnlf l<cant 


.416 


.456 


None 


Insignificant 


.406 { 


.440 


None 


Insignificant 


.497 ) 


- .535 


None . 


Insignificant 


.487 I 


.507 


None 


Insignificant 


.443 


.'428 


.015 


High 


.542 


.522 


.020 


High 



Variable (Teacher's. Responses) 

Change In basic skills teaching^ 
Change in affective development 
Change in dealing with special ■ 

.learning problems 
Improvement in student behavior .. 
Change in classroom organization 
Improvement in student achievement 
Continuation of project methods 
Percent project. goals achieved 

NOTE: Entries are based on a regression of each variable on a vector of dichoto- 
mous variables, each corresponding to an identifier of a school in the survey. r2 
from this basic regression is shown in the second column. 

att_ . ■ 

Increment Due to School" is the difference between the variance explained in 
the test regression (r2) and the that could occur by chance if the Independent 
variables had no explanatory power beyond that of the project repressions (reported 
in Table B.4). Since the increment required to establish statistical significance 
will vary depending on the variance explained by the project regressions, the appro- 
priate critical value is reported in" the third column. This value of r2 corresponds 
to the statistical test of whether F(56, 298) 1.38. for p < 0.05. ' 



Table B-6 

SUMMARY- OF VARIANCE EXPLAINED IN DEPENDENT VARIABLES . / 



Project Effect 





Insignificant 


Moderate 


High 


Very High 


u 

JjJ Inslgniei^ant 

iM 
IM- 

w 




Change In basic skills teaching 
Change in dealing with 

affective development 
Change in dealing with special 

learning problems 
Intprovement in student behavior 


Classroom organizaCfon change 
Improvement in student 
achievement 




O ))oderace 
O 










•g High 

CO 




Continuation of project methods 


Percent project goals achieved 




Very high 
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their dealing with students' individual learning problems showed some 
similarity among teachers in the same project, bu>t the similarity did 
not come close to dominating these classroom outcomes. The same find- 
ing holds for improvements in studen-f behavior. Being in a particular 
project simply did hot determine the bulk of these project, outcomes 
for .teachers. This suggests that research investigating innovations 
and their -results should analyze individual variations among teachers. 
Summarizing t^he results of an innovation for all of the teachers in 
a prpject ignores and throws away a wealth of^ information pertaining 
to-' the large individual component. of innovative outcomes. ■ , 

Project Effects on Achievement and Classroom Organization 

Teachers' judgments about the change in their classroom orga- 
ni'Zation practices and their assessment of improvements in student 
achievement showed a high level of systematic response by teachers 
in the same project (though there was no additional similarity of re- 
sponses for teachers in the same school). The extreme difficulty of 
classroom organization projects probably explains the coherence of 
teachers' responses to this measure.; a project that did not intend to 
change classroom organization would probably produce uniformly low 
reports of classroom change, while projects attempting major classroom 
organization changes evidently had sufficient impact and novelty to 
pi^oduce a Common perception of change for their teachers. Simila*rly, 
it is possible that the emphasis (or lack of it) on student achieve-, 
ment gains by district policymakers may contribute to a consistency 
in teachers' efforts to improve achievement. 

Continuation — A School Phenomenon 

Teachers' continuation of project methods showed a high level of 
similarity among teachers in the same school and moderate similarity 
fpr teachecs within a project. This result lends credence, to our 
data-analysis approach, of examining the classroom level separately 
from the district level. 
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Project and School Effects on Implement at ion Effectiveness 

Teachers' reports of percent project goals achieved showed a 
high tendency to conform within particular projects and within schools 
even after taking account of project efffei^ts. When teachers evaluated 

the project as a whole (rather .than their own experiences as partici- 

A- 

pants in the project), , they tended to gravitate' toward the opinions 
of their colleagues. " . . " * ' 
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Appendix C 
PARENT INVOLVEMENT 



\ 
\ 



Unlike the other Titles of ESEA- — specifically, Title I and 
Title VII — Title III grants carry no requirement to- involve parents 
in project operation or governance. When parents were involved in 
Title III projects, their involyement was p'art of the local district 
staff's strategy to assist implementation. In our sample, approxi- 
mately 25 percent of the teachers stated that parent involvement was 
a major goal, for their projects."^ It can be inferred from the distri- 
bution of teacher responses to the survey question that parent involve- 
ment was by no means a "popular" or a routinely elected implemefltation 
strategy; but when it was made a major project goal, it was regarded 
as ,,Sn Important part of project operations—not as an incidental 
'activity., 'as is so often the case in Title I projects, 

. Furthermore ,', our analysis suggests that if project staff chose 
parent involvement as a goal, (or, as we inferred above, if projects 
really wanted to involve parents), they usually succeeded in doing so. 

We asked teacheris to what' extent the strategies used by their projects 

2 ■ ' 

were effective in promoting parent participation. The correlation 



The specific question- asked of teachers and the distribution of 
the 1010 responses were: (Parent Involvement as a Goal) To what 
.extent was parent involvement a major goal, of this, projeot?^^ 

7.1% 8.0% 9.4% 13.2% 11.7% 17.7% 33.0% 

I I \ L I : I J 

7 6 " 5 4.3 21 

^ To a very ^. Not at all ■ 

large extent ^ 

2 

The specific question and, the distribution of the 533 responses 
were: (Success in Involving Parents) *^How ^effeative were these strat- 
egies in involving parents?^* 

9.0% 14.1% 20.5% 23.4% 15.0% 13.2% 4.9% 

I I I f' I i I 

7 6 5 4 / 3 2 1 

Very Not at all 

effective * . effective 
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between this question and the question concerning parent involvement 
as a major project goal is approximately .76, indicating tl^at those 
projects genuinely wanting parent involvement were usually able to 
devise strategies to obtain it. In the past, the apparent failure 
of many Title I and other projects to promote parent ,involvemen^r has 
been blamed on "parent apathy'^; one inference of our analysis, however, 
i^ that- "project apafhy" is more to blame, because many proj ec ts' did 
-not really want to involve, parents in the first plage. Our analysis 
suggests that the effectiveness of patent involvement strategies, liKe 
that of many other educational strategies, depends on a. point of view 
about parents' role in the governance and operation of school affairs — 
a point of view that is independent of mandates or guidelines. 

It is not enough merely to want parent involvement. Strategies 
to gain it must be devised, and some are much more effective than. 

3 

'others. Teachers were asked what strategies their projects had used. 
We examined the overall relationship between their resp' . < ♦•n this 
question and their ratings of how effective the strategies had been. 
The resulting partial R of .-26 is high enough to suggest that strategies 
per se do matter, bu^: is low. enough to indicate that a great deal de- 
pends on the strategy choisen. ^. ■ ^ 
Of the seven strategies identified, parent training was found to 

be the most effective. Parent involvement /In classroom activities — 

■ ■ ' • /■ 

■ ■ .' / ■ ■ , .. ■ , 

/ . 

The specific survey question and the distribution of responses^, 

were: (Parent' Involvement Strategies) Which of the following strat- 
egies y if any, was used to involve parents in the project?^'. 

No. of 

%^ Responses 

\ No parent involvement strategy 58.1 385 

\ Parent training ••• •• 26.9 ' 177 

*\ Parents hired as paid aides..... . ; . 26.6 175 

Other parent involvement in the classroom .. . 38.9 257 

Home visits by teacher 16.4 107 * 

Parent participation in project planning 

or decisionmaking 2(5.3 173' 

Parent participation in school planning 

or decisionmaking 20.4 134 

Other (please specify) 21.8 143 



other than as paid aides.— was also very effective. Together these 
two strategies corroborate the importance of active parent jjarticipa- 
tion, a theme that has recurred repeatedly during the past decade *s 
experience. with these strategies. Most failures to engage parents ' 
have been failures by design,, with parents relegated to the ..role, of 
passive ."recipients," in which they have little power over program 
operation or the educational process. Such strategies are typically ' 
variations on the PTA or newsletter model: Inf orm^at'ion about project 
operations is dispensed, and decisions are per functorily reviewed with 
parents. Parents have largely ignored pro forma efforts of this kind, 
.-especially low-income parents for whom time is at a premium. . 

The success of pa-rent training in our sainple of Title III projects 
suggests that most parents, not merely parents of the "disadvantaged,'*, 
are eager to learn what they can do to further their chiljdren's educa- 
tion- Parent training enables them to acquire and apply educational 
skills in the home or classroom. 

In short, effective parent involvement strategies are those that 
p.arents perceive as supporting their active role in their children's 
education and providing information about more effective parenting. 
Organizing and carrying out* these strategies require considerable 
. .staff time and effort — effort that .is unlikely to be expended unless 
• project staff members genuinely value parent involvement. Consequently, 
the relationship between parent involvement as a goal and effectiveness 
in achieving parent participation is not surprising. . • 

How does parent involvement relate to project outcomes? Tradi- 
tionally, parents have been "silent partners'* in education, arid PTA 
meetings . have often been more social affairs than working sessions. 
Against this background, a serious and successf ul" eTf ort to involve 
..parents in project operat ions s ignif ies a decided change in point of 
view concerning . the governance* and conduct of school affairs; in such- 
cases, one would expect to encounter attempts to alter '^till other 
"traditional" aspects of classroom management and^ practice , such as 
: student/teacher relations or teacher roles. Our analysis suppoi^s this 
oroposition. Projects that had parent involvement as a major goal also 
.resulted in s ignif ic an til y more teacher change in a nymber.of areas:.. 

' ' .244 
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teachers' ability, to recognize and solve individual students ' spec ial 
learning problems; teachers' c lassroom organization techniques ; and 
teachers' ability to work with students in affective development. ^ 
Parent involvement did not alter the way in which teachers taught 
basic skills."^^ • It seems likely that projects pursuing parent involve- 
ment were less concerned with innovatiou in particular subject areas 
*■••*" ' ' ■ . 

than with a more general change in school or classroom climate and 

management. Thus, it is not unexpected that parent involvement was. 
also positively related to change in student attitudes or "behavior , 
but hot to student achievement. 

Parent involvement as a project goal was positively related to 
continuation of project methods^ but not to the continuation of project 
materials y . after the end of federal funding. This finding supports 
the notion that act ive . parent involvement is part of a point of view 
about the delivery of educational services — an- approach to education 
that assumes considerable change in traditional attitudes and therefore 
requires considerable initial, coimnitment on the part of staff . - 
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